15 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: Avoiding hour creep: get your work done and still go home at 5PM

You want to work 40 hours a week, you want to head home at 5PM, but-there's this bug. And you're stuck, and you really need to fix it, and you're in the middle so you work just a little longer. Next thing you know you're leaving work at 6PM.

And before long you're working 50 hours a week, and then 60 hours a week, and if you stop working overtime it'll hit your output, and then your manager will have a talk with you but how you really need to put in more effort. So now you're burning out, and you're not sure what you can do about it.

But what if you were more productive?

What if you knew how to get your work done on company time, and could spent your own time on whatever you wanted?

You can-with a little time management.

Some caveats

Before we get to the actual techniques you'll be using, some preliminaries.

First, these techniques will only work if you have a manager who judges you based on results, not hours in the office. Keep in mind that there are many managers who claim they want a 50-hour workweek, but in practice will be happy if you do a good job in just 40. I'm also assuming your company is not in constant crisis mode. If these assumptions are wrong, better time management won't help: it's time to find another job.

Second, these techniques are here to help you in day-to-day time management. If production is down, you may need to work longer hours. (And again, if production is down every week, it's time to find another job.)

Finally, for simplicity's sake I'm assuming you get in at 9:00AM and want to leave at 5:PM. Adjust the times below accordingly if you start later in the day.

Taking control over your time

Since your problem is time creep, the solution is hard limits on when you can start new work-together with time allocated to planning so future work is more productive.

Here's the short version of a schedule that will help you do more in less time:

  1. When you get in to work you read your checkpoint from the previous workday (I'll explain this in a bit).
  2. Until 3:30PM you work as you normally would.
  3. After 3:30PM you continue on any existing task you're already working on. If you finish that task you can start new tasks only if you know they will take 15 minutes or less. If you don't have any suitable tasks you should spend this time planning future work.
  4. At 4:45PM you stop what you're doing and checkpoint your work.
  5. At 5:00PM you go home.

Let's delve deeper so you can understand what to do, and why this will help you.

End of day → start of next day: checkpointing

In the last 15 minutes of your day you stop working and checkpoint your work. That is, you write down everything you need to know to get started quickly the next morning when you come to work.

If you're in the middle of a task, for example, you can check in "XXX" comments into your code with notes on the next changes you were planning to make. If you're doing planning, you can assign yourself a task and write down as much as possible about how you should implement it.

This has two benefits:

  1. Next morning when you get to work, and even more so after a weekend or vacation, you'll spend much less time context swapping and trying to remember where you were. Instead, you'll have clear notes about what to do next.
  2. By planning your work for the next day, you're setting up your brain to work out the problem in the background, while you're enjoying your free time. You're more likely to wake up in the morning with a solution to a hard problem, or have an insight in the shower. For more about this see Rich Hickey's talk on Hammock Driven-Development.

No new large tasks after 3:30PM

By the time the afternoon rolls by you've been working for quite a few hours, and your brain isn't going to work as well. If you're in the middle of a task you can keep working on it, but if you finish a task you should stop taking on large new tasks near the end of the day. You'll do much better starting them the next day, when you're less tired and have a longer stretch of time to work on them.

How should you spend your time? You can focus on small tasks, like code reviews.

Even more importantly, you can spend your afternoon doing planning:

In the long run planning will make your implementation work faster. And by limiting planning to only part of your day you're making sure you don't spend all of your time planning.

Going home at 5:00PM exactly

There's nothing inherently wrong with spending a few more minutes finishing something past 5:00PM. The problem is that you're experiencing hour creep-it's a problem for you specifically. Having a hard and fast rule about when you leave will force you not to stay until 6:00 or 7:00PM.

Plus, sometimes it's not just a few minutes, sometimes you'll need more than that to solve the problem. And a task that will take two hours in the evening might take you only 10 minutes in the morning, when you're well-rested.

In the long run you'll be more productive by not working long hours.

A recap

Here's a recap of how you should be spending your day at work:

There's nothing magic about this particular set of rules, of course. You will likely want change or customize this plan to your own needs and situation.

Nonetheless, since you are suffering from hour creep I suggest following this particular plan for a couple of weeks just so you start getting a sense of the benefits. Once you've taken control over your time you can start modifying the rules to suit your needs better.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

15 Jun 2018 4:00am GMT

08 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: The true meaning of unit testing

You probably already know what "unit testing" means. So do I.

But-what if our definitions are different? Does unit testing mean:

I've seen both definitions used quite broadly. For example, the Python standard library has a unittest module intended for generic automated testing.

So we have two different definitions of unit testing: which one is correct?

Not just unit testing

You could argue that your particular definition is the correct one, and that other programmers should just learn to use the right terminology. But this seems to be a broader problem that applies to other forms of testing.

There's "functional testing":

Or "regression testing":

Why is it so hard to have a consistent meaning for testing terms?

Testing as a magic formula

Imagine you're a web developer trying to test a HTTP-based interaction with very simple underlying logic. Your thought process might go like this:

  1. "Unit testing is very important, I should unit test this code-that means I should test each function in isolation."
  2. "But, oh, it's quite difficult to test each function individually… I'd have to simulate a whole web framework! Not to mention the logic is either framework logic or pretty trivial, and I really want to be testing the external HTTP interaction."
  3. "Oh, I know, I'll just write a test that sends an HTTP request and make assertions about the HTTP response."
  4. "Hooray! I have unit tested my application."

You go off and share what you've learned-and then get scolded for not doing real unit testing, for failing to use the correct magic formula. "This is not unit testing! Where are your mocks? Why are you running a whole web server?"

The problem here is that the belief that one particular kind of testing is a magic formula for software quality. "Unit testing is the answer!" "The testing pyramid must be followed!"

When a particular formula proves not quite relevant to our particular project, our practical side kicks in and we tweak the formula until it actually does what we need. The terminology stays the same, however, even as the technique changes. But of course whether or not it's Authentic Unit Testing™ is irrelevant: what really matters is whether it's useful testing.

A better terminology

There is no universal criteria for code quality; it can only be judged in the context of a particular project's goals. Rather than starting with your favorite testing technique, your starting point should be your goals. You can then use your goals to determine, and explain, what kind of testing you need.

For example, imagine you are trying to implement realistic looking ocean waves for a video game. What is the goal of your testing?

"My testing should ensure the waves look real."

How would you do that? Not with automated tests. You're going to have to look at the rendered graphics, and then ask some other humans to look at it. If you're going to name this form of testing you might call it "looks-good-to-a-human testing."

Or consider that simple web application discussed above. You can call that "external HTTP contract testing."

It's more cumbersome than "unit testing," "end-to-end testing," "automated testing", or "acceptance testing"-but so much more informative. If you told a colleague about it they would know why you were testing, and they'd have a pretty good idea of how you were doing the testing.

Next time you're thinking or talking about testing don't talk about "unit testing" or "end-to-end testing." Instead, talk about your goals: what the testing is validating or verifying. Eventually you might reach the point of talking about particular testing techniques. But if you start with your goals you are much more likely both to be understood and to reach for the appropriate tools for your task.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

08 Jun 2018 4:00am GMT

03 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: Get productive faster at your new job—without working overtime

You've just started a new job and you're not productive yet: you get stuck, you need help, you don't know enough. You need to learn a new codebase, a new set of conventions, a new set of business problems. You might even need to learn a new programming language.

And so now you feel anxious-

Are you doing well enough?

Are you producing enough code?

How is your manager feeling about your progress?

It's natural to make yourself feel more comfortable by working overtime. You're showing your manager that you're trying, at least, and by working long hours you might get a little bit more done. You don't want to work overtime in the long run, of course, but you can worry about that in the future.

Unfortunately, working long hours is-as you might suspect-the wrong solution: at best it won't help, and it might even make your situation worse. Let's see why overtime isn't helpful, and then move on to a better solution: a solution that will make you more productive and make you look good to your manager.

Long hours won't solve your problem

Working overtime might make you feel a little better. Unfortunately it's also a bad solution in the short run, and a big problem in the long run.

In the short run, you're not actually going to get more done. Long hours will just tire you out, won't help you learn any faster, and pretty much are never the solution to producing more (here's some research if you don't believe me). Even worse, you might end up giving your manager the wrong impression: you're working long hours and you're still not productive yet?

In the long run, you're setting bad expectations about your work hours. If you have a mediocre manager, let alone a bad one, they will often expect you to keep working those long hours. You need to set boundaries from the start: "here are my work hours, I won't work more outside of emergencies."

There's a better solution: focusing on your real goal, which is learning everything you need to know about your new project.

The real solution: learning with feedback

You have two core problems:

  1. You need to learn a lot, and you don't necessarily even know what you need to learn.
  2. You can't demonstrate you're being productive to your manager the usual way, by fixing bugs or adding features.

You can solve both problems at once with the following process:

  1. Every Friday, with your week's work still fresh in your mind, write down:
    • Everything you've learned that week.
    • What you think you need to learn next.
  2. First thing Monday morning when you get back to work, send an email to your manager with what you wrote Friday, and an additional question: "What is missing from this list? What else do I need to learn?"
  3. Your manager can now provide you with feedback about additional things you need to learn.
  4. When you get stuck and don't want to ask for help just yet, take a break and go learn something on your list.

If you follow this process:

And of course, no overtime required.

Want more suggestions for getting started on your best foot? Last time I started a new programming job I created a personal checklist: all the things I should be doing on my first few days at work. If you'd like to read it, you can download it here.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

03 Jun 2018 4:00am GMT

02 Jun 2018

feedPlanet Twisted

Moshe Zadka: Avoiding Private Methods

Assume MyClass._dangerous(self) is a private method. We could have implemented the same functionality without a private method as follows:

This might seem onerous, but consider that now, dangerous is part of the public interface of a class, and would need to be explicitly documented as to assumptions and guarantees. This documentation would have had to have been in comments around _dangerous anyway -- in order to clarify what its assumptions are, and what invariants it is violating in MyClass -- otherwise, maintaining the code that calls _dangerous would be hard.

Of course, this documentation is almost certain to be missing. The existence of _dangerous itself implies this was an almost mechanical refactoring of similar code into a method, with the excuse of "its private" used to avoid considering the invariants and interface.

Even if the documentation did exist, now it is possible to unit-test that the documentation is correct. Furthermore, if we use best practices when we define MyClass -- in other words, avoid creating an InnerClass object in the initializer, and only creating it in an MyClass.from_parameters, we are also in a good position to unit test MyClass.

This, of course, presented the worst case: the code for _dangerous touches absolutely every data member of MyClass. In real life, the worst case is not often encountered. When we look at a private method as a code smell, and contemplate the best way to refactor it away, it turns out that we often can find a coherent set of attributes that really does make sense as InnerClass on their own merits.

Credit: This is based on an off-handed comment Glyph made in his blog post about attrs. I am also grateful to him for reviewing a draft copy of this post, and making many useful suggestions. All mistakes in interpretation or explanation are mine alone.

02 Jun 2018 4:30am GMT

20 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: Staying focused, the productive way

Your manager keeps telling you that you're not getting enough done. So you decide to become more focused, since as everyone knows, to be a productive programmer you need to stay focused. Deep-diving into TV Tropes, chatting with your friends, or reading up on that fancy new web framework might be fun, often even educational, but they won't get that feature you're working on out the door.

So you get noise canceling headphones, and only read your email once a day, and use the Pomodoro technique, and became laser-focused on your code-but still, you're not productive enough. Your colleague across the hall doesn't write code faster than you, and yet somehow they make more of an impact, they get things more done. You know it, and your manager knows it.


Because staying focused is not enough to make you productive. In fact, it's often the other way around: staying focused is a side-effect of what truly makes you productive.

A short visit to a military jail

Imagine a yard full of dirty gravel, and mixed in with the gravel are tiny twigs, trash, and the like. How long could you spend crawling around looking for this debris before you'd get bored? How long could you stay focused?

Long ago I lived in Israel, and as a Jewish citizen I was required to serve three years in the military. For a variety of reasons, personal and political, I had no interest in becoming a soldier, and so I attempted to avoid conscription by getting a medical discharge for mental health reasons. While on the base I was part of a transients' unit on the military base: we would clean bathrooms and the like while awaiting processing.

As our story unfolds, I was having a very bad day. My attempt to get a discharge was failing, as the military psychiatrist had decided there was nothing wrong me. And to make things worse, the sergeant who ran the unit wanted me to go off and do some work on the base, and I couldn't deal with it.

So I said "no"-which to say, I refused orders, serious business in the military. The sergeant organized a quick trial, and the officer in charge sentenced me to a day in the on-base jail. Perhaps for entertainment, perhaps to enforce the importance of obeying orders, while I was in the jail my guards ordered me to search for little bits of tiny debris that were mixed in the jailyard's gravel.

And so I spent quite a while, crawling around on my knees in the rain, working hard at a pointless task. The guards were impressed, and eventually they felt bad enough to give me an umbrella to keep the rain off.

The moral of the story

I started this episode by refusing to work, and refusing work that had some purpose (washing dishes, or cleaning a bathroom). I ended by working hard doing something that was a complete waste of time.

Both choices were good ones, because in both cases I was working towards my goals:

  1. My broadest goal was getting kicked out of the military. Cooperating was doing me no favors: spending some time in jail for refusing orders demonstrated I was not going to be a good soldier.
  2. My secondary goal was minimizing the amount of time I spent in jail. I had met a soldier on base who had spent his time in jail getting in trouble with his guards, so he'd been sentenced to even more time. He ended up spending months on a military prison base. I wanted to be a model prisoner, so I could get out of jail as quickly as possible.

Staying focused and avoiding distractions is all fine and good, so long as the work you're doing actually helps you achieve your goals. If it's not, you're staying focused on the wrong thing. I could have stayed focused by following orders-and that would have been the wrong way to achieve my goal of getting kicked out of the military.

Plus, knowing your goals can help you stay focused. If you don't care about your task, then you'll have a hard time focusing. But once you do understand why you're doing what you do, you'll have an easier time staying on task, and you'll have an easier time distinguishing between necessary subtasks and distracting digressions. And that's why I was able to enthusiastically clean debris from gravel.

This then is the key to achieving your goals, to productivity, and to staying focused: understanding your goals, and then working towards them as best you can.

Applying your goals to staying focused

So how do you use goals to stayed focused?

  1. Figure out the goals for your task.
  2. Strengthen your motivation.
  3. Judge each part of your work based on your goals.

1. Discovering your goals

Start with the big picture: why are you working this job? Your goals might include:

Then focus down on your particular task: why is this task necessary? It may be that to answer this question you'll need do more research, talking to the product owner who requested a feature, or the user who reported a bug. This research will, as an added bonus, also help you solve the problem more effectively.

Combine all of these and you will get a list of goals that applies to your particular task. For example, let's say you're working on a bug in a flight search engine. Your goals might be:

  1. Money: I work to make money.
  2. Organizational goal: I work here because I think helping people find cheap, convenient flights is worth doing.
  3. Task goal: This bug should be fixed because it prevents users from finding the most convenient flight on certain popular routes.
  4. Fun: This bug involves a challenging C++ problem I enjoy debugging.

2. Strengthening your motivation

Keeping your goals in mind will help you avoid distractions, and the more goals you're meeting, and the more your various goals point in the same direction, the better you'll do. If you have weak or contradictory goals then you can try different solutions:

3. Judging your work

Understanding your goals will not only help you avoid small distractions (noise, TV Tropes), but bigger distractions as well: digressions, seemingly useful tasks that shouldn't actually be worked on. Specifically, as you go about solving your task you can use your goals to judge whether a new potential subtask is worth doing.

Going back to the example above, imagine you encounter some interesting C++ language feature while working on it can be tempting to dive in. But judged by the four goals it will only serve the fourth goals, having fun, and likely won't further your other goals. So if the bug is urgent then you should probably wait until it's fixed to play around.

On the other hand, if you're working on a pointless feature, your sole goals might be "keep my manager happy so I can keep getting paid." If you have two days to do the task, and it'll only take two hours to implement it, spending some time getting "distracted" learning a technical skill might help with a different goal: switching to a more interesting position or job.

Start with your goals

Once you know goals, you can actually know what it takes to be productive, because you'll know what you're working towards. Once you know your goals, you can start thinking about how to avoid distractions because you'll know you're doing work that's worth doing.

Before you start a task, ask yourself: what are my goals? And don't start coding until you have an answer.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

20 May 2018 4:00am GMT

18 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: It's time to quit your shitty job

If it's been months since you had a day where you feel good-

If you hate getting out of bed in the morning because that means you'll have to go to work-

If your job is tiring you out so much you can't get through the day without a nap-

It's time to quit your shitty job. It's time to quit your shitty job and go someplace better, a job where a good night's sleep is all you need. A job where you're valued. A job where people don't shout at each other, or demean you, or destroy the project you've put all your energy into.

But quitting can be difficult: you have a sense of commitment, the fear of change, the indecision about whether your job is really that shitty. So to help you make your decision, and quit in the best possible way, in the rest of this post I will cover:

  1. Identifying a shitty job.
  2. Whether you should quit (spoiler: yes).
  3. Preparations you should make before quitting: legal, bureaucratic, social.
  4. When to quit.
  5. How you should quit.

(Note that some of this will be US-centric, since that's where I live and what I know best.)

Identifying a shitty job

Shitty jobs can be surprisingly hard to identify.

Sometimes this is because you don't have a reasonable baseline, or the shittiness has become normalized through exposure. I've heard of companies with the following symptoms, for example, and I would consider either grounds for immediately starting a search for a better job:

Another reason you might not notice you have a shitty job is a subtle shift over time. A good job slowly gets worse, and your existing relationships and loyalty blind you to the symptoms-for a while, anyway. You might be forced to reconsider due to:

I could go on with other examples, but there are two core themes here:

  1. Your company doesn't value its employees.
  2. You don't trust company management in the aggregate.

Again, this may not always have been the case. You may trust many of your managers, and know that they value you and your coworkers. But things change, and not always for the better: what matters is the way the company is now, and who has power now, not the way it used to be.

Should you quit your shitty job?


But you should do so at the right time, and with a little preparation.

When should you quit your shitty job?

Ideally, you should have another job lined up before you quit.

I once had to give notice of quitting unexpectedly, without prior planning. A more observant coworker gave notice the same day, but they had started looking a couple months before, when we had a round of layoffs. So while I spent a couple months not getting paid, they moved straight on to another job. The lesson: it pays to look for early signs of shittiness, so that you can leave in the best possible way.

Once you realize you have a shitty job, you should start interviewing elsewhere. Having an existing job improves your negotiation position, since you always have the implicit alternative offer of staying where you are. Two offers you can play against each other, or "I'm far along in interview process with another company" is better, but lacking that you need to downplay how shitty your current job is.

You'll want a break to catch your breath and relax in between jobs: you can easily negotiate a couple of weeks time off in between jobs. A month shouldn't be much harder to get.

In practice, your job may be so awful that it leaves you with no time or energy to look for another job. In this case you might be forced to quit without a new job lined up. You can prepare for this by living below your means and saving some money.

Preparing for quitting

Here are some things you should do before quitting any job:

At a shitty job you may also need to make copies of some documents: specifically, any emails or other documents where promises are made to you re pay, benefits, and so on. Once you've been paid what you're owed and you've left your job, you won't need those anymore and they should be deleted or shredded. But when it's your last day at work and you're trying to get the back pay they owe you, you want to make sure you have documentation.

Speaking of back pay, if you work for a company that has an "unlimited vacation" policy, take some vacation before you quit. You're not going to get paid for those vacation days you haven't taken. (In general, if a company has "unlimited vacations" I recommend taking lots of vacation throughout the year, since it's use or lose it.)

How to quit

It's a shitty job, and you may be utterly relieved at leaving it, but-you should quit politely. Your management may simply be misguided, or suffering under pressures you don't understand (VCs in cover-your-ass mode can be quite destructive). Your manager might grow as a person. Your co-workers might end up working with you again.

So just give your notice, with the smallest possible amount of time you have to stay there. You can tell close coworkers why you're leaving (they probably already know). And on your last day of work just leave, quietly and politely.

For a while you will feel sad: those projects will never get finished. But mostly you will feel relief.

It's time-

-time to quit your shitty job.

As I mentioned above, I once made the mistake of hanging on when I shouldn't have, unlike a more clued-in coworker. (You can hear the whole story by signing up for my Software Clown newsletter, where I share 20+ years of my mistakes so that you can avoid them.)

Don't make my mistake. I had to quit anyway, and without the benefit of advance planning or having a job lined up. Start looking for a new job now, while you're still able to hold on-your job probably probably won't be getting any better.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

18 May 2018 4:00am GMT

17 May 2018

feedPlanet Twisted

Jonathan Lange: Announcing quay-admin

We use quay.io a fair bit at work-all our internal Docker images are stored there. I like it a lot, but the website makes it really hard to see who can access your repositories.

In particular, if someone ever leaves your organization, you have to click through all of your repositories one at a time to see whether they have been granted access to a repository as an individual, rather than as a member of a team. This might be OK if you have two or three repositories, but not if you have hundreds.

I had some spare time today, so I wrote a tool to help with this. It's called quay-admin and you can install it now:

$ pip install quayadmin

This will give you a command-line tool called quay-admin that you can run to see which users outside of your organization have access to your repositories.

I originally tried to write it in Go, basing it off my colleague's excellent quay-exporter project-a tool that turns security vulnerability warnings into Prometheus metrics so you can get alerted. Unfortunately, getting Go to work well with Swagger APIs is a bit fiddly, and I didn't have that much spare time. So I tried Python, knowing that it has excellent libraries for working with RESTful services.

First cut used requests, which helped me figure out which APIs I needed and how they gave me the data I wanted. Next version used treq, which allowed me to parallelize, which saves precious seconds of my only life.

It's been an age since I've written Twisted code, but it all comes rushing back fairly quickly. I've found that I miss certain things from Haskell's async library, notably mapConcurrently, but they are easy enough to add.

Releasing Python code is way different though. At Glyph's recommendation, I tried flit, which seems to work OK.

Thanks to dstufft, glyph, dreid, AlexGaynor, wsanchez, and others who patiently answered my questions while I was writing this, and who in some cases wrote much of the actual software I am building on top of.

Thanks also to quay.io for actually publishing their API docs. It genuinely helps.

17 May 2018 11:00pm GMT

Moshe Zadka: PyCon US 2018 Twisted Birds of Feather Open Space Summary

We would like Twisted to support contextvars -- this would allow cross-async libraries, like eliot to do fancy things.

Klein is almost ready to be used as-is. Glyph has the good branch which adds

But it is too big, and we need to break it to reviewable pieces to add it to master.

The other option for a Twisted-native web framework is Cyclone. It is not under heavy development, but this is mostly because it is done and reasonably stable: Duo Security is using it in production.

We are slowly improving the Request object by taking it out of the built-in and reimplementing it externally. Wilfredo is doing it in a side-project.

We talked a little about advanced use cases: How do you use a reactor in a non-main thread? The only marginally documented installSignalHandlers argument does that just fine.

If you want to spread processing between multiple processes, Ampoule does that. Help is greatly appreciated.

If you want to do weird things with resources, Moshe did something on Twitch this one time.

We made sure everyone knows their help would be appreciated, and gameified: Review tickets and participate on the mailing list.

Remeber: the book Expert Twisted is available for pre-orders!

17 May 2018 1:50am GMT

16 May 2018

feedPlanet Twisted

Moshe Zadka: PyCon 2018 US Docker Birds of Feather Open Space Summary

We started out the conversation with talking about writing good Dockerfiles. There is no list of "best practices" yet. Hynek reiterated for us "ship applications, not build environments". Moshe summarized it as "don't put gcc in the deployed image."

We discussed a little bit what we are trying to achieve with better docker files. Shared base? Reproducible builds?

We talked about some of the challenges for building Docker on CI systems, especially from inside containers.

Docker on air-gapped machines is hard. So many parts assume free access to the internet.

We went on to discuss how to use multistage Dockerfiles. One important bit is what "installable artifact" to move. Some suggested wheels. Moshe suggested Pex. Hynek suggested copying a virtual environment, and Moshe showed an example

There was some discussion on making small images. The consensus was that Alpine is usually part of the answer.

There was a lot of discussion on the trade-offs between updating too soon, and too late. Some of the techniques to control update times were mentioned:

We talked about GPU containers, for machine learning. Apparently nvidia-docker is still nascent but works.

We talked about how to keep your registry clean. Unfortunately, the consensus is that you will need to build your own tooling.

We discussed what registries people use.

We touched lightly on performance. Docker can use either overlayfs vs devicemapper. It's complicated

Would you run your DB in Docker? Docker is just a packaging format. You can run Postgres in Docker just fine, and mount in the data directory. However, usually people are asking about using Orchestration Frameworks for that.

StatefulSets in K8s are sometimes useful for databases.

If you are running your dev DB in Docker, data is not important. In that case, consider using eatmydata to improve performance.

We all agreed you should never use the system Python for your applications. Then how do you get Python in your Docker image?

Finally, we discussed the ultimate heresy: running more than one process inside your container. Or is it? Moshe mentioned that anyone running uwsgi or gunicorn is already running a process manager: just one that happens to be part of the WSGI "binary". We mentioned supervisor and NColony for explicit process management.

16 May 2018 1:50am GMT

08 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: Guest Post: Networking for programmers with very little free time

The following guest post is by Moshe Zadka, explaining the importance of networking and how you can do it with minimal time outside work.

A good professional network is a long-term asset. When you're looking for a new job you can talk to people you know, ask them if their company is hiring, and then have them submit your resume directly to the hiring manager. This will allow you to skip the "resume filter", and often get you past the phone screen as well.

But even if a professional network is useful, how do you find the time to build it? You probably don't want to have to get out every evening to a social gathering, and spend hours talking, just to plan for a hypothetical future job.

One way to start building your professional network with little time spent is by focusing on your current job. Over time your colleagues will leave for other jobs, and every former colleague is a potential referral to another company. So you should always make sure you have non-work contact details for your colleagues.

Unfortunately, you can only expand your network so much from attrition at work: if you want a larger network, you will have to do some work. But by making judicious use of your time, and going to the right venues, you can grow and maintain a good professional network while still only spending one or two evenings a month at events.

How networking helped me

In one of the San Francisco Python meetups, I met someone working at PayPal- a company I had no interest working for. However, we kept in touch. At some point, he moved to a start-up. At another point, I found myself looking for a job, after a company shutdown. Because I had kept in touch with him, it was easy to reach out.

Even though I was on a tight timeline for getting another job, he made sure I was fast-tracked through the process - a pro-forma resume review, and skipping the phone screen. I still had to go through a half-day's worth of interview panel, but removing the simple filters from my path probably saved a week or more worth of being unemployed, and also let me put pressure on other prospective employers to fast-track me.

Business cards

Business cards sound like an antiquated thing, something you might see on "Mad Men". However, even with modern smartphones, there is no faster way to share your contact details with someone you've just met. For that, a business card's most important part is your e-mail address.

In all of the opportunities below, give people a business card when the conversation is done. Making your own cards is free to cheap nowadays, no need to wait for your job to print you one. In any case, you want to make sure you have your personal e-mail on the card, not your work e-mail.


If you are already employed, ask your job to send you to relevant conferences. Some places have a budget for "professional development", others have funds specifically marked "conferences" - or maybe it's under the recruiting budget. Choose a relevant conference, and remind your manager that sending you to the conference is a form of training: a great investment in employees.

Some companies will only fund your trip if you speak at a conference. Most conferences understand that some people will only come if they speak, and structure their timeline accordingly: you'll know whether your talk is accepted far enough in advance to get your manager to sign off on sending you. An efficient way to send talk proposals is to recycle - if a talk is declined from one conference, it is fine to send it to another closely related one, although sometimes it will have to be tweaked slightly. If the audiences are sufficiently distinct you can even reuse a talk you've already given.

Once you are at a conference, attend birds of feather sessions, and try to sit with new people for conference meals, if those are served. This is a great way to meet more people at the conference. Giving a talk is also a great way to meet more people: you can often meet other speakers, and many people in the audience will want to talk to you afterwards.


Many places have tech meetups in the evening. You can probably find time to go to a meetup once a month. If you do go, make sure to make the most of your time - mingle, talk and hand out your business card.

Avoid going to the same meetup month after month - while it is comfortable, it tends to be the same people: your goal should be to expand your network. So once you stop meeting new people, switch to a new meetup.

As with conferences, giving a talk at a meetup is great way to meet new people. You might even be able to work on your talk at work, if you can pitch it as a recruiting event to your manager.

Engineering blog

If your company has an engineering blog, participate. Find something you have done recently which was interesting or surprising, and write about that experience.

If your company does not have an engineering blog, see if you can make one happen. It helps with recruiting, and helps people develop in their career.

Keeping in touch

Keep in touch with your network. If you come across an article relevant to someone, send it to them with a note "thought it might be interesting". Often they will already have read it, but will be interested in sharing their thoughts.


Developing and maintaining a professional network does not need a huge time investment - a little bit goes a long way, if properly allocated. And when the day comes that you need a new job, that small investment will pay off. While you usually can't just get a job just by knowing someone, a network will help skip past companies "resume filters", and you can have a more streamlined interview process if you have a friend on the inside.

Moshe is a core developer on the Twisted project, and has been using Python for 20 years. He's just published a book, "from python import better".

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

08 May 2018 4:00am GMT

04 May 2018

feedPlanet Twisted

Jonathan Lange: Site updates

I am pleased to announce that the recent TLS certificate problems and outages to jml.io have been fully resolved.

Here are some notes on what happened and what I did about it.


jml.io is a statically generated blog that's hosted on AWS. The HTML pages are generated locally and uploaded to S3 buckets. These buckets are then served by CloudFront, which acts as a CDN. The domain names are managed on Route53.

Before now, these were managed by clicking around in the AWS console. There might have been a time when they were generated by Ansible playbooks, but I can't find the playbooks anymore.

A couple of weeks ago, the wildcard TLS certificate for jml.io expired. This meant that anyone who browsed to the site with a modern browser got a scary warning saying the site wasn't trustworthy. And fair enough too!

To get the site working, I needed to get a new certificate and distribute it from AWS. Recalling the steps to do this properly was too hard, and besides I knew of a better way.

Enter Terraform

We use Terraform to manage our AWS infrastructure at my employer, and really quite like it. I personally have some qualms about HCL, its configuration language, that I might write about later, but I like both it and Terraform more than any alternatives I'm aware of.

Because my site was down and because I really don't have time to do considered maintenance, I decided to migrate the whole thing to Terraform. This would mean that all of my thinking and decisions could be stored in a Git repo, rather than my memory.

I would also use the AWS Certificate Manager to generate and manage the certificates, sparing me the difficulty of purchasing, storing, configuring, and later renewing them myself.

How it happened

I spent the first week snatching the occasional hour here and there figuring out how Terraform worked. While we use it at Weaveworks, I wasn't the one to set it all up, and editing something built by someone else is very different from being able to build to from scratch.

In particular, I needed to get a feel for the workflow and for how Terraform's means of abstraction, modules, actually worked.

The next week I started migrating all the "redirect" buckets to Terraform. These are S3 buckets for my old domain names (code.mumak.net, mumak.net, etc.) that now redirect here.

Doing this involved figuring out how to import things from Terraform, how to use modules, how to edit Terraform state when you've refactored something.

It's quite slow going. The terraform plan step takes quite a while, which means the edit/test loop is bit of a grind. This really hurts when you are snatching a half-hour before bed here and there to get things done.

During this process, I got a bunch of excellent advice and working, reusable Terraform code from David Reid.

Once I got the redirect buckets incorporated, I moved on to their DNS records. That went fairly smoothly if slowly.

Then I decided to set up CloudFront, ACM, and a Lambda for HSTS all at once. It would have worked great, except that all my stuff was on us-west-2, and all the cool features for integrating with CloudFront are in us-east-1.

So today I had the joy of migrating buckets from one AWS region to another. AWS has no built-in support for this that I know of, so the way you do it is create a new bucket in the new region, copy all the content over, delete the old bucket, then wait a while for eventually consistent data stores and/or batch jobs to do their thing, then create a bucket in the new region with the old name, then copy all the data over again, then delete the temporary bucket.

It's a real hassle, and AWS's silly global bucket namespace thing adds an edge of frission: what if someone steals my name while I'm waiting to retry?

The new setup

Everything's on AWS, managed by Terraform. Even the Terraform state and lock are kept there. I haven't set up anything like terradiff yet, but it's only a matter of time.

The module set up means I've got a pretty clear list of what's a genuine static website and what's a redirect site. There's some duplication, but its mostly of boilerplate rather than of magic strings.

Going forward, I'm going to use the extra automation provided by Terraform to make publishing to this blog a bit easier for me. I think I can also take some of the stuff I've learned and incorporate it into our work infrastructure.


Terraform is great for managing AWS stuff. AWS is a pretty cool way of hosting a static site if you care about TLS certificates (which you should). dreid is awesome for giving me so much useful help at the right time.

04 May 2018 11:00pm GMT

02 May 2018

feedPlanet Twisted

Moshe Zadka: Wheels

Announcment: My book, from python import better, has been published. This post is based on one of the chapters from it.

When Python started out, one of the oft-touted benefits was "batteries included!". Gone were the days of searching for which XML parsing library was the best -- just use the one built-in to the standard library. However, the standard library can only hold so much special purpose stuff. Few now remember, but it used to have SGI Audio specific functionality.

These days, one of the biggest benefits of Python is the extensive third-party repository of stuff. This is the Python Package Index (PyPI), formerly known as the "Cheese Shop" after an obscure Monty Python skit. Of course, what else would be available from the Cheese shop than wheels of cheese? But a second pun was hiding behind the term "wheels": those are the things that need no reinvention!

The new PyPI warehouse launched, with new code hosting unbelievable amounts of content: around 140,000 packages at times of this post (unless I take too long in publishing it, and then who knows how big PyPI will be!)

Nobody can sift through 140K package descriptions, of course. A short-lived attempt to have "Stars" fell victim to allegations of ballot stuffing and moderation, and was quickly removed. Searching on key words would be useful, but searching without sorting rarely is -- and what would you sort on?

PyPI is not the place to find which libraries are useful. It is the place to find objective truths: which version is the latest, when was it released, what is a project's homepage, etc. Recommendations are best found elsewhere.

The first place I like to start is with the Awesome Python list. However, it is important to note that its contribution guidelines are just "submit a link" and there is no official way to remove a library from the list. Thus, the "awesome" in the name means "someone once thought it was awesome, and cared enough to add it". The list should be treated as mild suggestions. Before using a library, check release history, GitHub health, code quality and other metrics you might care about.

Another useful resource is Planet Python. It is a feed aggregator of various blogs. Many of the blog posts will feature either a recommendation of a particular library, a release announcement, or just discussion which involves using a third-party library. Along side the written word is the live performance -- PyVideo links to more Python talks than you can shake a stick at, aggregating talks from conferences around the world: again, many of the talks will feature discussion of a particular third-party library.

Last, but not least, the live, interactive, version of PyVideo: Python meetups and conferences. Those are where I discovered some of my favorite libraries.

02 May 2018 3:00pm GMT

01 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: A refurbished iPad, the CAP theorem, and a lesson on negotiation

We'll get to the iPad and CAP theorem soon, but first, let's talk about negotiation:

And that's just the way things are, and it's not like there's anything you can do, right?

Maybe. But quite possibly there is something you can do.

To explain why, I'd like to share an edifying tale, a story of broken promises and ultimate-albeit minor-triumph. Along the way we'll take a detour into distributed systems theory, and when we're done there will even be a moral (hint: it's about negotiation).

A purchase is made

Once upon a time, at a different job that subsidized such things, my wife purchased an iPad. Years passed, operating systems were upgraded, and over time this iPad became too slow to run some apps, and too old to run others.

It was time to buy a new iPad.

The day Apple released their new 2018 iPad we went to Apple's online store and purchased a refurbished 2017 model. We got a confirmation email, and looked forward to a tablet that could keep up with many companies' unwillingness to ship performant code (looking at you, Skype).

The next day Apple sent us another email: our order had been canceled, and our money would be refunded. When we checked the store, the refurbished model was no longer in stock.

Our somewhat cheaper iPad was not coming.

A theory is introduced, with some references to distributed systems

Why did the Apple Store cancel our order? Perhaps it was a bug, but I have another theory: the constraints of the CAP theorem.

Eric Brewer's CAP theorem states that a distributed data store-a system composed of multiple nodes-can only have two out of three properties:

  1. Consistency: all nodes have same view of the data.
  2. Availability: the system can respond successfully to clients.
  3. Partition-tolerance: if the network between the nodes fails, the system can continue to operate.

Now, the online Apple Store is quite likely to be a distributed system, given the need to scale to many users. And it needs to store data, the size of the inventory of each item in the store. Given a choice between those three properties, the only two reasonable choices are availability and partition-tolerance.

It's far better to have a store that is available than to have completely consistent tracking of inventory. There is a cost to this choice, though: every once in a while a large rush of orders will cause inconsistent views of the available inventory.

Because the system can't enforce consistency, the same iPad is sold to two people. What to do?

One common solution (alas, I can't find the original paper where I read the idea) is "compensation" or "apology": out-of-band business processes to repair the mistakes. In this case, a post-processing stage that notices the double-sold iPad and handles it somehow.

How this rare but inevitable mistake is handled is a policy decision, and Apple's chosen policy is to simply cancel the order-contrary to a guarantee they make on their website.

A complaint is made

If you go to the Apple Store website's refurbished section, you will see in small letters at the top that "availability is guaranteed once we receive your full payment." Given that promise, and the fact we'd gotten a confirmation email for our payment, my wife called up customer service and politely asked why our order was canceled.

The representative went off, and after some delay she indicated that she'd talked to her manager and she'd gotten approval to send us a new iPad instead. So the next week we received a 2018 iPad, while only paying the cost of a refurbished 2017 model.

Success, and easy success at that that. My wife didn't have to complain loudly, point out Apple was in the wrong, or hassle anyone. She just asked.

The topic of negotiation is reintroduced

Apple made a promise (payment == guaranteed delivery), and then violated it. Why? Violating the guarantee on their website was an opening offer in a negotiation.

When you're negotiating, you need to ask for what you want, or you won't get it. In practice most people won't ask and won't complain, and so it's in Apple's interest to start with a low offer: most people will get the email canceling the order, grumble a bit, and re-order something else.

Often how you ask is also important: you need to ask the right way. I once had a $5000 medical expense denied over and over by a health insurance company. Eventually I indicated I wished to file a grievance-a bureaucratic procedure I found on their website-and suddenly they found a "coding error" that had caused the problem and my bill was paid. (My uninformed guess is that grievances get reported to state regulators.)

The same dynamic is part of how many companies take advantage of employees, and the cost you'll pay for not asking is even higher. When a company gives you a job offer they will have a salary range they are willing to pay, but they will usually not offer their high number. Instead, they will offer you the lowest number possible they think you might accept. Sometimes that number will also be far below market rate.

Many people will simply accept that initial salary offer-I've certainly made that mistake-and therefore end up getting paid much less than they could otherwise get. But of course that's just an initial offer, and simply asking for more will usually result in a higher offer. And you can do even better if you do a little work beforehand.

A summary is presented

Here's what we've learned:

  1. Recognize negotiating situations: you've already lost if you don't realize you're negotiating.
  2. Ask for what you want, or you won't get it.
  3. Ask in the right way.

For a useful guide to negotiating your salary and benefits in general, see Valerie Aurora's negotiating guide. And if it's your working environment you care about-working shorter hours, working from home, and the like-check out The Programmer's Guide to a Sane Workweek.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

01 May 2018 4:00am GMT

29 Apr 2018

feedPlanet Twisted

Twisted Matrix Laboratories: Twisted 18.4.0 Released

On behalf of Twisted Matrix Laboratories, I'm honoured to announce the release of Twisted 18.4.0!

The highlights of the release are:

For more information, check the NEWS file (link provided below).

You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-18.4.0/NEWS.rst>.

Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted!

Twisted Regards,
Amber Brown (HawkOwl)

29 Apr 2018 11:19am GMT

09 Apr 2018

feedPlanet Twisted

Itamar Turner-Trauring: Programming as natural ability, and the bandaid of long work hours

Your project deadline is getting closer and closer, and you're stuck: you don't know what to do. Your manager won't help, they just push you to work evenings and weekends-and when it looks like you're going to fail, they hand the project over to another programmer.

You've failed your manager, and there's a little voice in the back of your head telling you that maybe you're missing what it takes to be a programmer, maybe you just don't have the requisite natural ability.

That little voice is lying to you.

It's the other way around: your manager has failed you, and is compounding the failure by conveying a destructive mindset, what's known as a fixed mindset. To understand what I'm talking about, let's take a quick detour into the psychology of education, and then return to those long hours you've been working.

Growth mindset vs fixed mindset

Carol Dweck, a professor of psychology at Stanford University, contrasts two mindsets when learning:

According to Dweck's research, students with a growth mindset do better than students with a fixed mindset. A fixed mindset is self-defeating: it keeps you from learning, and it keeps you from even trying to learn.

In the context of programming this suggests that starting with the attitude that programming is a set of skills, skills that you can learn, will result in a much better learning experience.

But what does this have to do with long working hours?

You can't work smarter, so you gotta work longer

In an article that was one of the inspirations for this post, Carol Dweck points out that a common failure mode among educators is to praise effort, working harder, instead of praising learning. While they may claim to be encouraging a growth mindset, they are simply perpetuating a fixed mindset.

This failure mode appears in the software world as well. Let's assume for the moment that programming is a natural ability: just before we're born, the Angel of Software injects between 0 and 100 milliliters of Programming Juice into our soul. If you're really lucky, you might even get 110ml!

Now, given that each and every one of us only has a limited amount of Programming Juice, how can you maximize our output? You can't learn more, so there's no way to do things more efficiently. You can't improve your skills, so there's no way to become more productive.

So what's left?

All together now: WORK LONGER HOURS!

Working longer ain't smart

The truth, of course, is that there is no Angel of Software, there is no Programming Juice. Programming is just a bunch of skills. You can learn those skills, and so can most anyone else, given motivation, time, support, and some good teachers. And you can become more and more productive by continuing to learn.

If you believe in fixed talent, if you believe you can't improve, you won't try to learn. Long hours will be the only tool left to you.

When faced with a problem: just work longer hours.

When faced with another problem: work even longer.

You'll work and work and work, and you'll produce far less than you would have if you'd spent all that time improving your skills. And eventually you'll hit a problem you can't solve, and that you will never solve by working longer hours.

A growth mindset will serve you far better. You need to believe that skills can grow, and then you need to actually do the work to learn more and grow your skills.

And when you fail-and you will fail, because we all fail on occasion-take this as another opportunity to learn: look for the patterns and cues you should have have spotted. Having learned your lesson, next time you'll do better.

Is your job taking way all your personal time and freedom? You can succeed as a software engineer without working crazy hours.

09 Apr 2018 4:00am GMT

03 Apr 2018

feedPlanet Twisted

Moshe Zadka: Web Development for the 21st Century

(Thanks to Glyph Lefkowitz for some of the inspiration for this port, and to Mahmoud Hashemi for helpful comments and suggestions. All mistakes and issues that remain are mine alone.)

The Python REPL has always been touted as one of Python's greatest strengths. With Jupyter, Jupyter Lab in its latest incarnation, the REPL has been lifted into the 21st century. It has become the IDE of the future: interactive, great history and logging -- and best of all, you can use it right from your browser, regardless of your platform!

However, we still have 20th century practices for developing web applications. Indeed, the only development is that instead of "CTRL-c, up-arrow, return", we now have "development servers" which are not "production ready" support auto-reloading -- the equivalent of a robot doing "CTRL-c, up-arrow, return".

Using the REPL to develop web applications is pure bliss. Just like using it to develop more linear code: we write a function, test it ad-hocly, see the results, and tweak.

When we are sufficiently pleased, we can then edit the resulting notebook file into a Python module -- which we import from the next version of the notebook, in order to continue the incremental development. Is such a thing possible?

Let's start by initializing Twisted, since this has to happen early on.

from tornado.platform.twisted import install
reactor = install()

Whew! Can't forget that! Now with this out of the way, let's do the most boring part of every Python program: the imports.

from twisted.web import server
from twisted.internet import endpoints, defer
import klein
import treq

Now, let's start Klein app. There are several steps involved here.

root = klein.app.resource()

We take the Klein resource object...

site = server.Site(root)

...make a wrapping site...

ep = endpoints.serverFromString(reactor, "tcp:8000")

...create an endpoint...

<Deferred at 0x7f54c5702080 current result: <<class 'twisted.internet.tcp.Port'> of <class 'twisted.web.server.Site'> on 8000>>

and like "Harry met Sally", eventually bring the two together for Klein to respond on port 8000. We have not written any application code, so Klein is currently "empty".

What does that mean?

async def test_slash():
    response = await treq.get('http://localhost:8000')
    content = await response.content()
    return content

This function uses Python 3's async/await features, to use treq (Twisted Requests) and return the result. We can use it as our ad-hoc debugger (but we could also use a web browser -- this is naturally hard to show in a post, though).

<Deferred at 0x7f54c5532630>
b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">n<title>404 Not Found</title>n<h1>Not Found</h1>n<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>n'

Ah, yeah. Even / gives a 404 error -- we have literally defined nothing. OK, this is easy to fix:

def something_useful(request):
    return 'Hello world'

Wait, did this work?

<Deferred at 0x7f54c53d8d30>
b'Hello world'

Yep. But it's not a proper sentence...woops.

def something_useful(request):
    return 'Hello, world!'

Nice. Punctuation. Force. Determination. Other nouns.

Did it change anything?

<Deferred at 0x7f54c4b9e240>
b'Hello, world!'


Incremental web application development. Without an "auto-loading" "not production grade" server.

We took advantage of several incidental issues. The Jupyter kernel is Tornado based. Twisted has both a production-grade web development framework, and the ability to run on top of the tornado loop. The combination is powerful.

03 Apr 2018 4:30am GMT