16 Aug 2018

feedPlanet Twisted

Itamar Turner-Trauring: How to say "no" to your boss, your boss's boss, and even the CEO

You've got plenty of work to do already, when your boss (or their boss, or the CEO) comes by and asks you to do yet another task. If you take yet another task on you're going to be working long hours, or delivering your code late, and someone is going to be unhappy.

You don't want to say no to your boss (let alone the CEO!). You don't want to say yes and spend your weekend working.

What do you do? How do you keep everyone happy?

What you need is your management to trust your judgment. If they did, you could focus on the important work, the work that really matters. And when you had to say "no", your boss (or the CEO!) would listen and let you continue on your current task.

To get there, you don't immediately say "no", and don't immediately say "yes".

Here's what you do instead:

  1. Start with your organizational and project goals.
  2. Listen and ask questions.
  3. Make a decision.
  4. Communicate your decision in terms of organizational and project goals.

Step 1: Start with you goals

If you want people to listen to you, you need a strong understanding of why you're doing the work you're doing.

You should be able to connect your individual action to project success, and connect that to organizational success. For example, "Our goal is to increase recurring revenue, customer churn is too high and it's decreasing revenue, so I am working on this bug because it's making our product unusable for at least 7% of users."

When you're just starting out as an employee this can be difficult to do, but as you grow in experience you can and should make sure you understand this.

(Starting with your goals is useful in other ways as well, e.g. helping you stay focused).

Step 2: Listen and ask questions

Your lead developer/product manager/team mate/CEO/CTO had just stopped by your desk and given you a new task. No doubt you already have many existing tasks. How should you handle this situation?

To begin with, don't immediately give an answer:

Instead of immediately agreeing or disagreeing to do the task, take the time find out why the task needs to be done. Make sure you demonstrate you actually care about the request and are seriously considering it.

That means first, listening to what they have to say.

And second, asking some questions: why does this need to be done? What is the deadline? How important is it to them?

Sometimes the CEO will come by and ask for something they don't really care about: they only want you to do it if you have the spare time. Sometimes your summer intern will come by and point out a problem that turns out to be a critical production-impacting bug.

You won't know unless you listen, and ask questions to find out what's really going on.

Step 3: Decide based on your goals

Is the new task more important to project and organizational goals than your current task? You should probably switch to working on it.

Is the new task less important? You don't want to do it.

Not sure? Ask more questions.

Still not sure? Talk to your manager about it: "Can I get back to you in a bit about this? I need to talk this over with Jane."

Step 4: Communicate your decision

Once you've made a decision, you need to communicate it in a meaningful, respectful way, and in a way that reflects organizational and project goals.

If you decided to take the task on:

  1. Tell the person asking you that you'll take it on.
  2. Explain to the people who requested your previous tasks that those tasks will be late. Make sure it's clear why you took on a new task: "That feature is going to have to wait: it's fairly low on the priority list, and the CEO asked me to throw together a demo for the sales meeting on Friday."

If you decided not to take it on:

  1. Explain why you're not going to do it, in the context of project and organizational goals. "That's a great feature idea, and I'd love to do it, but this bug is breaking the app for 10% of our customers and so I really need to focus on getting it done."
  2. Provide an alternative, which can include:
    • Deflection: "Why don't you talk to the product manager about this?"
    • Queuing: "Why don't you add it to the backlog, and we can see if we have time to do it next sprint?"
    • Promise: "I'll do it next, as soon as I'm done with my current task."
    • Reminder: "Can you remind me again in a couple of weeks?"
    • Different solution: "Your original proposal would take me too long, given the release-blocker backlog, but maybe if we did this other thing instead I could fit it in. It seems like it would get us 80% of the functionality in a fraction of the time-what do you say?"

Becoming a more valuable employee

Saying "no" the right way makes you more valuable, because it ensures you're working on important tasks.

It also ensures your managers know you're more valuable, because you've communicated that:

  1. You've carefully and respectfully considered their request.
  2. You've taken existing requests you're already working on into account.
  3. You've made a decision not based on personal whim, but on your best understanding of what is important to your project and organization.

Best of all, saying "no" the right way means no evenings or weekends spent working on tasks that don't really need doing.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

16 Aug 2018 4:00am GMT

10 Aug 2018

feedPlanet Twisted

Itamar Turner-Trauring: There's always more work to do—but you still don't need to work long hours

Have you ever wished you could reduce your working hours, or even just limit yourself to 40 hours a week, but came up against all the work that just needs doing? There's always more work to do, always more bugs, always some feature that's important to someone-

How can you limit yourself to 40 hours a week, let alone a shorter workweek, given all this work?

The answer: by planning ahead. And planning ahead the right way.

The wrong way to plan

I was interviewing for a job at a startup, and my first interviewer was the VP of Engineering. He explained that he'd read my blog posts about the importance of work/life balance, and he just wanted to be upfront about the fact they were working 50-60 hours each week. And this wasn't a short-term emergency: in fact, they were going to be working long hours for months.

I politely noted that I felt good prioritization and planning could often reduce the need for long hours.

The VP explained the problem: they'd planned all their tasks in detail. But then-to their surprise-an important customer asked for more features, and that blew through their schedule, which is why they needed to work long hours.

I kept my mouth shut and went through the interview process. But I didn't take the job.

Here's what's wrong with this approach:

  1. Important customers asking for more features should not be a surprise. Customers ask for changes, this is how it goes.
  2. More broadly, the original schedule was apparently created with the presumption that everything would go perfectly. In the real world nothing ever goes perfectly.
  3. When it became clear that that there was too much work to do, their solution was to work longer hours, even though research suggests that longer hours do not increase output over the long term.

The better way: prioritization and padding

So how do you keep yourself from blowing through your schedule without working long hours?

  1. Prioritize your work.
  2. Leave some padding in your schedule for unexpected events.
  3. Set your deadlines shorter than they need to be.
  4. If you run out of time, drop the least important work.

1. Prioritize your work

Not all work is created equal. By starting with your goals, you can divide tasks into three buckets:

  1. Critical to your project's success.
  2. Really nice to have-but not critical.
  3. Clearly not necessary.

Start by dropping the third category, and minimizing the second. You'll have to say "no" sometimes, but if you don't say "no" you'll never get anything delivered on time.

2. Leave some padding in your schedule

You need to assume that things will go wrong and you'll need extra time to do any given task. And you need to assume other important tasks will also become critical; you don't know which, but this always happens. So never give your estimate as the actual delivery date: always pad it with extra time for unexpected difficulties and unexpected interruptions.

If you think a task will take a day, promise to deliver it in three days.

3. Set shorter deadlines for yourself

Your own internal deadline, the one you don't communicate to your boss or customer, should be shorter than your estimate. If you think a task will take a day, try to finish it in less time.


4. When you run out of time, drop the less important work

Inevitably things will still go wrong and you'll find yourself running low on time. Now's the time to drop all the nice-to-haves, and rethink whether everything you thought was critical really is (quite often, it's not).

Long hours are the wrong solution

Whenever you feel yourself with too much work to do, go back and apply these principles: underpromise, limit your own time, prioritize ruthlessly. With practice you'll learn how to deliver the results that really matter-without working long hours.

When you've reached that point, you can work a normal 40-hour workweek without worrying. Or even better, you can start thinking about negotiating a 3-day weekend.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

10 Aug 2018 4:00am GMT

09 Aug 2018

feedPlanet Twisted

Hynek Schlawack: Hardening Your Web Server’s SSL Ciphers

There are many wordy articles on configuring your web server's TLS ciphers. This is not one of them. Instead I will share a configuration which is both compatible enough for today's needs and scores a straight "A" on Qualys's SSL Server Test.

09 Aug 2018 6:00pm GMT

03 Aug 2018

feedPlanet Twisted

Moshe Zadka: Tests Should Fail

(Thanks to Avy Faingezicht and Donald Stufft for giving me encouragement and feedback. All mistakes that remain are mine.)

"eyes have they, but they see not" -- Psalms, 135:16

Eyes are expensive to maintain. They require protection from the elements, constant lubrication, behavioral adaptations to protect them and more. However, they give us a benefit. They allow us to see: to detect differences in the environment. Eyes register different signals when looking at an unripe fruit and when looking at a ripe fruit. This allows us to eat the ripe fruit, and wait for the unripe fruit to ripen: to behave differently, in a way that ultimately furthers our goals (eat yummy fruits).

If our eyes did not get different signals that influenced our behavior, they would not be cost effective. Evolution is a harsh mistress, and the eyes would be quickly gone if the signals from them were not valuable.

Writing tests is expensive. It takes time to write them, time to review them, time to modify them as code evolves. A test that never fails is like an eye that cannot see: it always sends the same signal, "eat that fruit!". In order to be valuable, a test must be able to fail, and that failure must modify our behavior.

The only way to be sure that a test can fail is to see it fail. Test-driven-development does it by writing tests that fail before modifying the code. But even when not using TDD, making sure that tests fail is important. Before checking in, break your code. Best of all is to break the code in a way that would be realistic for a maintenance programmer to do. Then run the tests. See them fail. Check it in to the branch, and watch CI fail. Make sure that this CI failure is clearly communicated: something big must be red, and merging should be impossible, or at least require using a clearly visible "override switch".

If there is no code modification that makes the test fail, of if such code modification is weird or unrealistic, it is not a good test. If a test failure does not halt the CI with a visible message, it is not a good CI. These are false gods, with eyes that do not see, and mouths that do not speak.

Real tests have failures.

03 Aug 2018 5:30am GMT

Moshe Zadka: Thank you, Guido

When I was in my early 20s, I was OK at programming, but I definitely didn't like it. Then, one evening, I read the Python tutorial. That evening changed my mind. I woke up the next morning, like Neo in the matrix, and knew Python.

I was doing statistics at the time. Python, with Numeric, was a powerful tool. It definitely could do things that SPSS could only dream about. Suddenly, something has happened that never happened before -- I started to enjoy programming.

I had to spend six years in the desert of programming in languages that were not Python, before my work place, and soon afterwards the world, realized what an amazing tool Python is. I have not had to struggle to find a Python position since.

I started with Python 1.4. I have grew up with Python. Now I am...no longer in my 20s, and Python version 3.7 was recently released.

I owe much of my career, many of my friends, and much of my hobby time to that one evening, sitting down and reading the Python tutorial -- and to the man who made the language and wrote the first version of that tutorial, Guido van Rossum.

Python, like all open source projects, like, indeed, all software projects, is not a one man show. A whole team, with changing personnel, works on core Python and its ecosystem. But it was all started by Guido.

As Guido is stepping down to take a less active role in Python's future, I want to offer my eternal gratitude. For my amazing career, for my friends, for my hobby. Thank you, Guido van Rossum. Your contribution to humanity, and to this one human in particular, is hard to overestimate.

03 Aug 2018 4:30am GMT

29 Jul 2018

feedPlanet Twisted

Itamar Turner-Trauring: Bad at whiteboard puzzles? You can still get a programming job

Practicing algorithm puzzles stresses you out: just looking at a copy of Cracking the Coding Interview makes you feel nervous.

Interviewing is worse. When you do interview you freeze up: you don't have IDE error checking and auto-completion, you can't use a search engine Google like a real programmer would, there's a stranger staring you down. You screw up, you make typos, you don't know what to say, you make a bad impression.

If this happens to you, it's not your fault! Whiteboard puzzles are a bad way to hire programmers.

They're not realistic: unless you're Jeff Goldblum haxoring the alien mothership's computer just in time for Will Smith to blow up some invaders, you're probably not coding on a 5-minute deadline.

And the skills they're testing aren't used by 95% of programmers 95% of the time. I recently had to do a graph traversal in dependency order-which meant I was all prepared to find my algorithms text book from college. But then I found this library already had a utility called toposort, and vague memories of classes 19 years ago reminded me that this was called a "topological sort". I didn't actually have to implement it, but if I did would have done it with textbook in hand, over the course of a couple of hours (gotta write tests!).

Unfortunately, many companies still use them, and you need a job. A programming job. What should you do?

Here are some ideas to help you find a job-even if you hate whiteboard puzzles.

1. Interview at companies with a better process

Not all companies do on-the-spot programming puzzles. The last three companies I worked at didn't-one had a take-home exercise that wasn't about algorithms (a decision I was involved in, and which I now regret because of the burden it puts on people with no free time). Two others just had talking interviews: I talked about myself, they talked about the company, all very relaxes and civilized.

To find such companies:

  1. Here's one list of 500+ companies that don't do whiteboard puzzles.
  2. The invaluable Key Values job board also tells you about the interview process at the covered companies (see the column on the right when looking at a particular company).

2. Offer an alternative

If you are interviewing at a company with whiteboard puzzles, you don't have to accept their process without pushing back. Just like your salary and working hours, the interview process is also something you can negotiate.

If you have some code you've written that you're particularly proud of and have the ability to share, ask the company if you can share it with them in lieu of a whiteboard puzzle. I once made the mistake of only suggesting this during the interview, and the guy who was interviewing me said he would have accepted it if I'd asked earlier. So make sure to suggest this before the day of the interview, so they have time to review the code in advance.

3. Take control of the process

If all else fails and you're stuck doing a puzzle, there are ways to take control of the process and make a good impression, even if the puzzle is too hard for you. I cover this in more detail in another post.

4. Don't give up

Finally, remember whiteboard puzzles have nothing to do with actual programming, even when the work you're doing is algorithmic. They're a hazing ritual you may be forced to go through, but they in no way reflect on your ability as a programmer.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

29 Jul 2018 4:00am GMT

13 Jul 2018

feedPlanet Twisted

Twisted Matrix Laboratories: Twisted 18.7.0 Released

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 18.7!

The highlights of this release are:
  • better support for async/await coroutines in regards to exception and traceback handling;
  • better support for reporting tracebacks in inlineCallbacks, now showing what you would expect in synchronous-like code
  • the epoll reactor now no longer hard-locks when running out of file descriptors
  • directory rendering in t.web works on Python 2 again
  • manhole's colouriser is better at handling Unicode
  • setting the groundwork for Python 3.7 support. Note that Python 3.7 is currently not a supported platform on any operating system, and may completely fail to install, especially on Windows.
For more information, check the NEWS file (link provided below).

You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-18.7.0/NEWS.rst>.

Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted!

Twisted Regards,
Amber Brown (HawkOwl)

13 Jul 2018 7:17pm GMT

10 Jul 2018

feedPlanet Twisted

Itamar Turner-Trauring: How to be judged by your output, not your time in the office

If you want to limit your working hours as a programmer you need to keep your boss happy. But what can you do if your boss seems to care more about your time in the office than how much you produce?

Not to mention the comparison to your other coworkers. If they fix ten bugs a week, but you only fix two, your boss might not be happy-even if the bugs you fixed had far more impact, and were much harder to address.

If you're stuck in this situation it may seem impossible to reduce your working hours. But before you give up and start looking for another job, it's worth seeing if you can improve the situation where you are.

Two reasons your boss might like long hours

If you're going to be judged by hours you need a manager who cares about the organization's or team's goals. There are two possibilities about how your boss is thinking:

Hours as proxy: Your boss cares about achieving goals, and is using hours as a mental shorthand for value produced. If you actually are a valuable worker, and make sure they know it, they won't notice your working hours, as in this real occurrence:

A programmer I know was having a conversation with their manager when the manager mentioned, in an offhand manner, that the company expected people to work 50 hours a week.

This programmer had always worked 40-45 hours a week, and the manager had never complained or noticed, because the programmer did good work. So the programmer kept their mouth shut, didn't comment, and kept on working their usual hours.

Hours as goal: Your boss may truly only care about hours in the office number of bugs fixed, or some other irrelevant measure. Which is to say, they're incompetent. In this case the suggestion that follows won't work, unless perhaps you can bypass your boss and reach someone who does care about organizational goals. Usually a job with different team or organization will serve you better.

Assuming your boss only uses hours as a proxy measure, let's see what you can do.

Starting with goals

It's 3PM on a Wednesday, and your boss swings by your desk and asks how things are going. You explain you're upgrading one of your JavaScript dependencies.

Your boss nods and wanders off, wondering if you're actually doing anything worthwhile. You have just wasted an opportunity to demonstrate your value.

What's the alternative? Starting with goals in mind.

Elsewhere I've talked about how starting with goals in mind will keep you focused, and is key to making you more productive. Starting with your organizational and team goals in mind can also help you both choose valuable work and explain its value to your boss.

For every task you work on, you should have a clear logical path from the big picture organizational goals, down to your team's goals, down to your project's goals, down to why this particular task at this particular time is a good way to advance those goals. If you can't make that connection, if you can't explain why you're doing what you're doing:

  1. You may not actually be doing anything valuable.
  2. Even if you are, you can't prove it.

Let's get back to that JavaScript dependency. If you started with goals in mind you might have decided this wasn't a particularly useful task to begin with, and worked on something else. Or, perhaps you know exactly why you're doing it.

In that case, the conversation might go something like this:

"You know how we've decided we wanted to increase user retention? Well, it looks like one of the problems is that our site is rendering way too slowly, so half our users bounce before the page finishes loading.

Turns out that font loading is the problem, and this library has a feature to fix that in its latest release. Once I've upgraded I should be able to get pages to render in a quarter of the time, and I'm hoping that'll increase user retention. And I have some other ideas in case that isn't sufficient."

Your boss goes away understanding that what you're doing is valuable. And if they're anywhere near competent they won't be thinking about the bug queue, or how many hours they've seen you in the office. They'll be thinking about the good work you're doing, and how they can tell their boss that the retention problem is being addressed.

Communicating your value based on goals

You can explain your work in this way when asked, but there's no reason not to do so proactively as well. Once a week, or once a month, you can take stock of what you've achieved and send an email to your boss. And you can also keep a copy of this email to update your resume when the time comes to look for a new job.

To recap:

  1. Understand why you're doing your work.
  2. Choose work that addresses those goals.
  3. Communicate to your boss why your work is helping those goals.

This will shift many managers from a hour mindset-driven by an assumption that your work isn't producing that much value-to a value mindset. Your boss will know your output is valuable, and as a result won't require the proxy measure of hours worked.

Learning how to work towards goals is, of course, easier said than done. It took me many years and many mistakes along the way. If you'd like to accelerate your learning, and take advantage of everything I've learned working as a programmer over the past 20 years, you can sign up for my Software Clown weekly newsletter. Every week I share a mistake and what you can learn from it.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

10 Jul 2018 4:00am GMT

03 Jul 2018

feedPlanet Twisted

Itamar Turner-Trauring: Five ways to work 35 hours (or less!) a week

You're tired of working yourself to death. You've had enough of the pressure of long hours, and you want not more money but more time: you want a shorter workweek. You want to work 35 hours a week, or 32, or even less.

But in an industry where some companies brag about their 70-hour workweeks, can you find a programming job with a short workweek?

The short answer is that yes, you can work 35 hours or less. I've done it at multiple jobs, and I know other programmers who do so as well.

The longer answer is that you have multiple different options, different ways of achieving this goal. Let's see what they are.

Option #1: Find a job that offers shorter hours

While they are few and far between, some organizations do offer shorter workweeks. For example, over on the excellent Key Values job board you can learn about Monograph, a company that provides a 32-hour workweek by default.

Option #2: Negotiate a custom deal

Just like you can negotiate a higher salary, you can also negotiate a shorter workweek. The best place to do it is at your current job, because you've likely got expensive-to-replace knowledge of business logic, organizational procedures, local tech stack, and so on. But you can also negotiate for a shorter workweek at a new job, if you do it right.

If you're interested in seeing how this is done, read my interview with a programmer who has been working part-time for 15 years.

Option #3: Become a consultant (the right way)

If you're a consultant and you do it right, you can raise your rates high enough that you don't need to work full time to make a living. Doing it right is important, though: if your hourly rate is low enough you're going to have to work long hours.

To learn about some of what it takes, Jonathan Stark's Value Pricing Bootcamp is one place to start.

Option #4: Start a product business (the right way)

If you're selling a product that you've created, the hours you work don't map one-to-one to your income. You have the upfront time for creating the product, and ongoing time for marketing and maintaining the product, but at that point you can sell the same product over and over with much smaller investment of time.

Consider for example Amy Hoy's explanation of how bootstrapping a business allowed her to make a living even with a chronic illness.

Option #5: Early retirement

Living below your means is a good idea in general: the more money you have in the bank the easier it'll be for you to find a better job, for example. But if you don't want to work at all, over the long term cutting your expenses can help you stop working altogether.

Liberate yourself from the office

One of the pernicious side-effects of the culture of long hours in tech is that even a 40-hour workweek seems impossible. But long hours aren't necessary: they're a crutch for bad management. Working shorter hours can actually make you more productive, productive enough that your total output goes up even with shorter hours.

In the short run, if you want to work fewer hours you have to do something about it.

In the long run, there's no reason why a 32-hour workweek couldn't be the standard-if we all push for it hard enough.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

03 Jul 2018 4:00am GMT

02 Jul 2018

feedPlanet Twisted

Moshe Zadka: Composition-oriented programming

A common way to expose an API in Python is as inheritance. Though many projects do that, there is a better way.

But first, let's see. How popular is inheritance-as-an-API, anyway?

Let's go to the Twisted website. Right at the center of the screen, at prime real-estate, we see:

What's there? The following is abridged:

class Echo(protocol.Protocol):
    def dataReceived(self, data):
class EchoFactory(protocol.Factory):
    def buildProtocol(self, addr):
        return Echo()

(This is part of an example on building an echo-server protocol.)

If you are wondering who came up with this amazing API, it is the same person who is writing the words you are reading. I certainly thought it was an amazing API!

Look at how many smart people agreed with me.

Django takes a page of tutorial to get there, but sure enough:

class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
class Choice(models.Model):
    question = models.ForeignKey(Question, on_delete=models.CASCADE)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)

Jupyter's echo kernel starts:

class EchoKernel(Kernel):
    implementation = 'Echo'
    implementation_version = '1.0'
    language = 'no-op'

Everyone is doing it. A project I have been a developer on for ~16 years. The most popular Python web library, responsible for who-knows-how-many requests per second in Instagram. A project that won the ACM award (and well deserved, at that).

However, popularity is not everything. This is not a good idea.

When exposing class inheritance as a public interface, that means committing to a level of backwards compatibility that is unheard of. Even adding private methods or attributes becomes dangerous.

Let's give a toy example:

class Writer:

    _write = lambda x: None

    def set_output(self, output):
        self._write = output.write

    def write(self, message):
        formatted = self.format(message)

    def format(self, message):
        raise NotImplementedError("format")

This is a simple writer, that, while initially sending everything down a black hole, can be set to write the output to a file-like object. It needs to format the messages, so the proper usage is to subclass and override format (while taking care not to define methods called set_output or _write.)

class BufferWriter(MultiWriter):

    _buffer = False

    def format(self, message):
        if self._buffer:
            return 'Buffer: ' + message
            return 'Message: ' + message

    def switch_buffer(self):
        self._buffer = not self._buffer

The simplest formatting would return the message as is. However, this formatter is slightly less trivial -- it prefixes the message with the word Buffer or Message, depending on an internal variable that can be switched.

Now we can do things like:

>>> bp = BufferWriter()
>>> bp.set_output(sys.stdout)
>>> bp.write("hello")
Message: hello
>>> bp.switch_buffer()
>>> bp.write("hello")
Buffer: hello

This looks good, so far. Of course, things are never so simple in real life. The writer library, naturally, gets thousands of stars on GitHub. It becomes popular. There's a development community, complete with a discord channel and a mailing list. So naturally, important features get added.

class Writer:

    _buffer = ""

    _write = lambda x: None

    def set_output(self, output):
        self._write = output.write

    def write(self, message):
        self._buffer += self.format(message)
        if len(self._buffer) > 10:
            self._buffer = ""

    def format(self, message):
        raise NotImplementedError("format")

Turns out people needed to buffer some of the shorter messages. This was a crucial performance improvement, that all users were clamoring for, so version 2018.6.1 is highly anticipated.

It breaks, though, the BufferWriter. The symptoms are weird: TypeError s and other such fun. All because both the superclass and the subclass are competing to access self._buffer.

With enough care, these problems can be avoided. A library which exposes classes for inheritance must add all new private methods or attributes as __ and, naturally, never ever add any public methods or attributes. Sadly, nobody does that.

So what's the alternative?

from zope import interface

class IFormatter(interface.Interface):

    def format(message):
        """show stuff"""

We define an abstract interface. This interface [1] has only one method -- format.

class Writer:

    _buffer = ""

    _write = lambda x: None

    _formatter = attr.ib()

    def set_output(self, output):
        self._write = output.write

    def write(self, message):
        self._buffer += self._formatter.format(message)
        if len(self._buffer) > 10:
            self._buffer = ""

We use the attrs library [#] to define our main functionality: a class that wraps other objects, which we expect to be IFormatter.

We can automatically verify, by instead having the _formatter line say:

_formatter = attr.ib(validator=lambda instance, attribute, value:
                               verify.verifyObject(IFormatter, value))

Note that this separates the concerns: the "fake method" format has moved to a "fake class" (an interface).

class BufferFormatter:

    _buffer = False

    def format(self, message):
        if self._buffer:
            return 'All Channels: ' + message
            return 'Limited Channels: ' + message

    def switch_buffer(self):
        self._buffer = not self._buffer

Note that now, if we only have the Writer object, there is no way to switch prefixes. Correctly switching prefixes means keeping access to the original object.

If there is a need to "call back" to the original methods, the original object can be passed in to the wrapped object. One advantage is that, being a distinct object, it is obvious one should only call into public methods and only access public variables.

Passing ourselves to a method is, in general, not an ideal practice. What we really should do, is to pass specific methods or variables directly into the method. But this is funny: when using inheritance, we always effectively pass ourselves to every method. So even this refactoring is a net improvement. When the biggest criticism of a refactoring is "this could now be improved even more", it usually means it is a good idea.


[1] The zope.interface library is a little like the abc libary: both give tools to clarify what methods we expect. However, the abc.ABC like inheritance a little too much. Glyph has a good explanation about the advantages.
[2] attrs makes defining Python classes much less boiler-platey. There's another Glyph post explaining why it is so good.

02 Jul 2018 5:00am GMT

15 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: Avoiding hour creep: get your work done and still go home at 5PM

You want to work 40 hours a week, you want to head home at 5PM, but-there's this bug. And you're stuck, and you really need to fix it, and you're in the middle so you work just a little longer. Next thing you know you're leaving work at 6PM.

And before long you're working 50 hours a week, and then 60 hours a week, and if you stop working overtime it'll hit your output, and then your manager will have a talk with you but how you really need to put in more effort. So now you're burning out, and you're not sure what you can do about it.

But what if you were more productive?

What if you knew how to get your work done on company time, and could spent your own time on whatever you wanted?

You can-with a little time management.

Some caveats

Before we get to the actual techniques you'll be using, some preliminaries.

First, these techniques will only work if you have a manager who judges you based on results, not hours in the office. Keep in mind that there are many managers who claim they want a 50-hour workweek, but in practice will be happy if you do a good job in just 40. I'm also assuming your company is not in constant crisis mode. If these assumptions are wrong, better time management won't help: it's time to find another job.

Second, these techniques are here to help you in day-to-day time management. If production is down, you may need to work longer hours. (And again, if production is down every week, it's time to find another job.)

Finally, for simplicity's sake I'm assuming you get in at 9:00AM and want to leave at 5:PM. Adjust the times below accordingly if you start later in the day.

Taking control over your time

Since your problem is time creep, the solution is hard limits on when you can start new work-together with time allocated to planning so future work is more productive.

Here's the short version of a schedule that will help you do more in less time:

  1. When you get in to work you read your checkpoint from the previous workday (I'll explain this in a bit).
  2. Until 3:30PM you work as you normally would.
  3. After 3:30PM you continue on any existing task you're already working on. If you finish that task you can start new tasks only if you know they will take 15 minutes or less. If you don't have any suitable tasks you should spend this time planning future work.
  4. At 4:45PM you stop what you're doing and checkpoint your work.
  5. At 5:00PM you go home.

Let's delve deeper so you can understand what to do, and why this will help you.

End of day → start of next day: checkpointing

In the last 15 minutes of your day you stop working and checkpoint your work. That is, you write down everything you need to know to get started quickly the next morning when you come to work.

If you're in the middle of a task, for example, you can check in "XXX" comments into your code with notes on the next changes you were planning to make. If you're doing planning, you can assign yourself a task and write down as much as possible about how you should implement it.

This has two benefits:

  1. Next morning when you get to work, and even more so after a weekend or vacation, you'll spend much less time context swapping and trying to remember where you were. Instead, you'll have clear notes about what to do next.
  2. By planning your work for the next day, you're setting up your brain to work out the problem in the background, while you're enjoying your free time. You're more likely to wake up in the morning with a solution to a hard problem, or have an insight in the shower. For more about this see Rich Hickey's talk on Hammock Driven-Development.

No new large tasks after 3:30PM

By the time the afternoon rolls by you've been working for quite a few hours, and your brain isn't going to work as well. If you're in the middle of a task you can keep working on it, but if you finish a task you should stop taking on large new tasks near the end of the day. You'll do much better starting them the next day, when you're less tired and have a longer stretch of time to work on them.

How should you spend your time? You can focus on small tasks, like code reviews.

Even more importantly, you can spend your afternoon doing planning:

In the long run planning will make your implementation work faster. And by limiting planning to only part of your day you're making sure you don't spend all of your time planning.

Going home at 5:00PM exactly

There's nothing inherently wrong with spending a few more minutes finishing something past 5:00PM. The problem is that you're experiencing hour creep-it's a problem for you specifically. Having a hard and fast rule about when you leave will force you not to stay until 6:00 or 7:00PM.

Plus, sometimes it's not just a few minutes, sometimes you'll need more than that to solve the problem. And a task that will take two hours in the evening might take you only 10 minutes in the morning, when you're well-rested.

In the long run you'll be more productive by not working long hours.

A recap

Here's a recap of how you should be spending your day at work:

There's nothing magic about this particular set of rules, of course. You will likely want change or customize this plan to your own needs and situation.

Nonetheless, since you are suffering from hour creep I suggest following this particular plan for a couple of weeks just so you start getting a sense of the benefits. Once you've taken control over your time you can start modifying the rules to suit your needs better.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

15 Jun 2018 4:00am GMT

08 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: The true meaning of unit testing

You probably already know what "unit testing" means. So do I.

But-what if our definitions are different? Does unit testing mean:

I've seen both definitions used quite broadly. For example, the Python standard library has a unittest module intended for generic automated testing.

So we have two different definitions of unit testing: which one is correct?

Not just unit testing

You could argue that your particular definition is the correct one, and that other programmers should just learn to use the right terminology. But this seems to be a broader problem that applies to other forms of testing.

There's "functional testing":

Or "regression testing":

Why is it so hard to have a consistent meaning for testing terms?

Testing as a magic formula

Imagine you're a web developer trying to test a HTTP-based interaction with very simple underlying logic. Your thought process might go like this:

  1. "Unit testing is very important, I should unit test this code-that means I should test each function in isolation."
  2. "But, oh, it's quite difficult to test each function individually… I'd have to simulate a whole web framework! Not to mention the logic is either framework logic or pretty trivial, and I really want to be testing the external HTTP interaction."
  3. "Oh, I know, I'll just write a test that sends an HTTP request and make assertions about the HTTP response."
  4. "Hooray! I have unit tested my application."

You go off and share what you've learned-and then get scolded for not doing real unit testing, for failing to use the correct magic formula. "This is not unit testing! Where are your mocks? Why are you running a whole web server?"

The problem here is that the belief that one particular kind of testing is a magic formula for software quality. "Unit testing is the answer!" "The testing pyramid must be followed!"

When a particular formula proves not quite relevant to our particular project, our practical side kicks in and we tweak the formula until it actually does what we need. The terminology stays the same, however, even as the technique changes. But of course whether or not it's Authentic Unit Testing™ is irrelevant: what really matters is whether it's useful testing.

A better terminology

There is no universal criteria for code quality; it can only be judged in the context of a particular project's goals. Rather than starting with your favorite testing technique, your starting point should be your goals. You can then use your goals to determine, and explain, what kind of testing you need.

For example, imagine you are trying to implement realistic looking ocean waves for a video game. What is the goal of your testing?

"My testing should ensure the waves look real."

How would you do that? Not with automated tests. You're going to have to look at the rendered graphics, and then ask some other humans to look at it. If you're going to name this form of testing you might call it "looks-good-to-a-human testing."

Or consider that simple web application discussed above. You can call that "external HTTP contract testing."

It's more cumbersome than "unit testing," "end-to-end testing," "automated testing", or "acceptance testing"-but so much more informative. If you told a colleague about it they would know why you were testing, and they'd have a pretty good idea of how you were doing the testing.

Next time you're thinking or talking about testing don't talk about "unit testing" or "end-to-end testing." Instead, talk about your goals: what the testing is validating or verifying. Eventually you might reach the point of talking about particular testing techniques. But if you start with your goals you are much more likely both to be understood and to reach for the appropriate tools for your task.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

08 Jun 2018 4:00am GMT

03 Jun 2018

feedPlanet Twisted

Itamar Turner-Trauring: Get productive faster at your new job—without working overtime

You've just started a new job and you're not productive yet: you get stuck, you need help, you don't know enough. You need to learn a new codebase, a new set of conventions, a new set of business problems. You might even need to learn a new programming language.

And so now you feel anxious-

Are you doing well enough?

Are you producing enough code?

How is your manager feeling about your progress?

It's natural to make yourself feel more comfortable by working overtime. You're showing your manager that you're trying, at least, and by working long hours you might get a little bit more done. You don't want to work overtime in the long run, of course, but you can worry about that in the future.

Unfortunately, working long hours is-as you might suspect-the wrong solution: at best it won't help, and it might even make your situation worse. Let's see why overtime isn't helpful, and then move on to a better solution: a solution that will make you more productive and make you look good to your manager.

Long hours won't solve your problem

Working overtime might make you feel a little better. Unfortunately it's also a bad solution in the short run, and a big problem in the long run.

In the short run, you're not actually going to get more done. Long hours will just tire you out, won't help you learn any faster, and pretty much are never the solution to producing more (here's some research if you don't believe me). Even worse, you might end up giving your manager the wrong impression: you're working long hours and you're still not productive yet?

In the long run, you're setting bad expectations about your work hours. If you have a mediocre manager, let alone a bad one, they will often expect you to keep working those long hours. You need to set boundaries from the start: "here are my work hours, I won't work more outside of emergencies."

There's a better solution: focusing on your real goal, which is learning everything you need to know about your new project.

The real solution: learning with feedback

You have two core problems:

  1. You need to learn a lot, and you don't necessarily even know what you need to learn.
  2. You can't demonstrate you're being productive to your manager the usual way, by fixing bugs or adding features.

You can solve both problems at once with the following process:

  1. Every Friday, with your week's work still fresh in your mind, write down:
    • Everything you've learned that week.
    • What you think you need to learn next.
  2. First thing Monday morning when you get back to work, send an email to your manager with what you wrote Friday, and an additional question: "What is missing from this list? What else do I need to learn?"
  3. Your manager can now provide you with feedback about additional things you need to learn.
  4. When you get stuck and don't want to ask for help just yet, take a break and go learn something on your list.

If you follow this process:

And of course, no overtime required.

Want more suggestions for getting started on your best foot? Last time I started a new programming job I created a personal checklist: all the things I should be doing on my first few days at work. If you'd like to read it, you can download it here.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

03 Jun 2018 4:00am GMT

02 Jun 2018

feedPlanet Twisted

Moshe Zadka: Avoiding Private Methods

Assume MyClass._dangerous(self) is a private method. We could have implemented the same functionality without a private method as follows:

This might seem onerous, but consider that now, dangerous is part of the public interface of a class, and would need to be explicitly documented as to assumptions and guarantees. This documentation would have had to have been in comments around _dangerous anyway -- in order to clarify what its assumptions are, and what invariants it is violating in MyClass -- otherwise, maintaining the code that calls _dangerous would be hard.

Of course, this documentation is almost certain to be missing. The existence of _dangerous itself implies this was an almost mechanical refactoring of similar code into a method, with the excuse of "its private" used to avoid considering the invariants and interface.

Even if the documentation did exist, now it is possible to unit-test that the documentation is correct. Furthermore, if we use best practices when we define MyClass -- in other words, avoid creating an InnerClass object in the initializer, and only creating it in an MyClass.from_parameters, we are also in a good position to unit test MyClass.

This, of course, presented the worst case: the code for _dangerous touches absolutely every data member of MyClass. In real life, the worst case is not often encountered. When we look at a private method as a code smell, and contemplate the best way to refactor it away, it turns out that we often can find a coherent set of attributes that really does make sense as InnerClass on their own merits.

Credit: This is based on an off-handed comment Glyph made in his blog post about attrs. I am also grateful to him for reviewing a draft copy of this post, and making many useful suggestions. All mistakes in interpretation or explanation are mine alone.

02 Jun 2018 4:30am GMT

20 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: Staying focused, the productive way

Your manager keeps telling you that you're not getting enough done. So you decide to become more focused, since as everyone knows, to be a productive programmer you need to stay focused. Deep-diving into TV Tropes, chatting with your friends, or reading up on that fancy new web framework might be fun, often even educational, but they won't get that feature you're working on out the door.

So you get noise canceling headphones, and only read your email once a day, and use the Pomodoro technique, and became laser-focused on your code-but still, you're not productive enough. Your colleague across the hall doesn't write code faster than you, and yet somehow they make more of an impact, they get things more done. You know it, and your manager knows it.


Because staying focused is not enough to make you productive. In fact, it's often the other way around: staying focused is a side-effect of what truly makes you productive.

A short visit to a military jail

Imagine a yard full of dirty gravel, and mixed in with the gravel are tiny twigs, trash, and the like. How long could you spend crawling around looking for this debris before you'd get bored? How long could you stay focused?

Long ago I lived in Israel, and as a Jewish citizen I was required to serve three years in the military. For a variety of reasons, personal and political, I had no interest in becoming a soldier, and so I attempted to avoid conscription by getting a medical discharge for mental health reasons. While on the base I was part of a transients' unit on the military base: we would clean bathrooms and the like while awaiting processing.

As our story unfolds, I was having a very bad day. My attempt to get a discharge was failing, as the military psychiatrist had decided there was nothing wrong me. And to make things worse, the sergeant who ran the unit wanted me to go off and do some work on the base, and I couldn't deal with it.

So I said "no"-which to say, I refused orders, serious business in the military. The sergeant organized a quick trial, and the officer in charge sentenced me to a day in the on-base jail. Perhaps for entertainment, perhaps to enforce the importance of obeying orders, while I was in the jail my guards ordered me to search for little bits of tiny debris that were mixed in the jailyard's gravel.

And so I spent quite a while, crawling around on my knees in the rain, working hard at a pointless task. The guards were impressed, and eventually they felt bad enough to give me an umbrella to keep the rain off.

The moral of the story

I started this episode by refusing to work, and refusing work that had some purpose (washing dishes, or cleaning a bathroom). I ended by working hard doing something that was a complete waste of time.

Both choices were good ones, because in both cases I was working towards my goals:

  1. My broadest goal was getting kicked out of the military. Cooperating was doing me no favors: spending some time in jail for refusing orders demonstrated I was not going to be a good soldier.
  2. My secondary goal was minimizing the amount of time I spent in jail. I had met a soldier on base who had spent his time in jail getting in trouble with his guards, so he'd been sentenced to even more time. He ended up spending months on a military prison base. I wanted to be a model prisoner, so I could get out of jail as quickly as possible.

Staying focused and avoiding distractions is all fine and good, so long as the work you're doing actually helps you achieve your goals. If it's not, you're staying focused on the wrong thing. I could have stayed focused by following orders-and that would have been the wrong way to achieve my goal of getting kicked out of the military.

Plus, knowing your goals can help you stay focused. If you don't care about your task, then you'll have a hard time focusing. But once you do understand why you're doing what you do, you'll have an easier time staying on task, and you'll have an easier time distinguishing between necessary subtasks and distracting digressions. And that's why I was able to enthusiastically clean debris from gravel.

This then is the key to achieving your goals, to productivity, and to staying focused: understanding your goals, and then working towards them as best you can.

Applying your goals to staying focused

So how do you use goals to stayed focused?

  1. Figure out the goals for your task.
  2. Strengthen your motivation.
  3. Judge each part of your work based on your goals.

1. Discovering your goals

Start with the big picture: why are you working this job? Your goals might include:

Then focus down on your particular task: why is this task necessary? It may be that to answer this question you'll need do more research, talking to the product owner who requested a feature, or the user who reported a bug. This research will, as an added bonus, also help you solve the problem more effectively.

Combine all of these and you will get a list of goals that applies to your particular task. For example, let's say you're working on a bug in a flight search engine. Your goals might be:

  1. Money: I work to make money.
  2. Organizational goal: I work here because I think helping people find cheap, convenient flights is worth doing.
  3. Task goal: This bug should be fixed because it prevents users from finding the most convenient flight on certain popular routes.
  4. Fun: This bug involves a challenging C++ problem I enjoy debugging.

2. Strengthening your motivation

Keeping your goals in mind will help you avoid distractions, and the more goals you're meeting, and the more your various goals point in the same direction, the better you'll do. If you have weak or contradictory goals then you can try different solutions:

3. Judging your work

Understanding your goals will not only help you avoid small distractions (noise, TV Tropes), but bigger distractions as well: digressions, seemingly useful tasks that shouldn't actually be worked on. Specifically, as you go about solving your task you can use your goals to judge whether a new potential subtask is worth doing.

Going back to the example above, imagine you encounter some interesting C++ language feature while working on it can be tempting to dive in. But judged by the four goals it will only serve the fourth goals, having fun, and likely won't further your other goals. So if the bug is urgent then you should probably wait until it's fixed to play around.

On the other hand, if you're working on a pointless feature, your sole goals might be "keep my manager happy so I can keep getting paid." If you have two days to do the task, and it'll only take two hours to implement it, spending some time getting "distracted" learning a technical skill might help with a different goal: switching to a more interesting position or job.

Start with your goals

Once you know goals, you can actually know what it takes to be productive, because you'll know what you're working towards. Once you know your goals, you can start thinking about how to avoid distractions because you'll know you're doing work that's worth doing.

Before you start a task, ask yourself: what are my goals? And don't start coding until you have an answer.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

20 May 2018 4:00am GMT

18 May 2018

feedPlanet Twisted

Itamar Turner-Trauring: It's time to quit your shitty job

If it's been months since you had a day where you feel good-

If you hate getting out of bed in the morning because that means you'll have to go to work-

If your job is tiring you out so much you can't get through the day without a nap-

It's time to quit your shitty job. It's time to quit your shitty job and go someplace better, a job where a good night's sleep is all you need. A job where you're valued. A job where people don't shout at each other, or demean you, or destroy the project you've put all your energy into.

But quitting can be difficult: you have a sense of commitment, the fear of change, the indecision about whether your job is really that shitty. So to help you make your decision, and quit in the best possible way, in the rest of this post I will cover:

  1. Identifying a shitty job.
  2. Whether you should quit (spoiler: yes).
  3. Preparations you should make before quitting: legal, bureaucratic, social.
  4. When to quit.
  5. How you should quit.

(Note that some of this will be US-centric, since that's where I live and what I know best.)

Identifying a shitty job

Shitty jobs can be surprisingly hard to identify.

Sometimes this is because you don't have a reasonable baseline, or the shittiness has become normalized through exposure. I've heard of companies with the following symptoms, for example, and I would consider either grounds for immediately starting a search for a better job:

Another reason you might not notice you have a shitty job is a subtle shift over time. A good job slowly gets worse, and your existing relationships and loyalty blind you to the symptoms-for a while, anyway. You might be forced to reconsider due to:

I could go on with other examples, but there are two core themes here:

  1. Your company doesn't value its employees.
  2. You don't trust company management in the aggregate.

Again, this may not always have been the case. You may trust many of your managers, and know that they value you and your coworkers. But things change, and not always for the better: what matters is the way the company is now, and who has power now, not the way it used to be.

Should you quit your shitty job?


But you should do so at the right time, and with a little preparation.

When should you quit your shitty job?

Ideally, you should have another job lined up before you quit.

I once had to give notice of quitting unexpectedly, without prior planning. A more observant coworker gave notice the same day, but they had started looking a couple months before, when we had a round of layoffs. So while I spent a couple months not getting paid, they moved straight on to another job. The lesson: it pays to look for early signs of shittiness, so that you can leave in the best possible way.

Once you realize you have a shitty job, you should start interviewing elsewhere. Having an existing job improves your negotiation position, since you always have the implicit alternative offer of staying where you are. Two offers you can play against each other, or "I'm far along in interview process with another company" is better, but lacking that you need to downplay how shitty your current job is.

You'll want a break to catch your breath and relax in between jobs: you can easily negotiate a couple of weeks time off in between jobs. A month shouldn't be much harder to get.

In practice, your job may be so awful that it leaves you with no time or energy to look for another job. In this case you might be forced to quit without a new job lined up. You can prepare for this by living below your means and saving some money.

Preparing for quitting

Here are some things you should do before quitting any job:

At a shitty job you may also need to make copies of some documents: specifically, any emails or other documents where promises are made to you re pay, benefits, and so on. Once you've been paid what you're owed and you've left your job, you won't need those anymore and they should be deleted or shredded. But when it's your last day at work and you're trying to get the back pay they owe you, you want to make sure you have documentation.

Speaking of back pay, if you work for a company that has an "unlimited vacation" policy, take some vacation before you quit. You're not going to get paid for those vacation days you haven't taken. (In general, if a company has "unlimited vacations" I recommend taking lots of vacation throughout the year, since it's use or lose it.)

How to quit

It's a shitty job, and you may be utterly relieved at leaving it, but-you should quit politely. Your management may simply be misguided, or suffering under pressures you don't understand (VCs in cover-your-ass mode can be quite destructive). Your manager might grow as a person. Your co-workers might end up working with you again.

So just give your notice, with the smallest possible amount of time you have to stay there. You can tell close coworkers why you're leaving (they probably already know). And on your last day of work just leave, quietly and politely.

For a while you will feel sad: those projects will never get finished. But mostly you will feel relief.

It's time-

-time to quit your shitty job.

As I mentioned above, I once made the mistake of hanging on when I shouldn't have, unlike a more clued-in coworker. (You can hear the whole story by signing up for my Software Clown newsletter, where I share 20+ years of my mistakes so that you can avoid them.)

Don't make my mistake. I had to quit anyway, and without the benefit of advance planning or having a job lined up. Start looking for a new job now, while you're still able to hold on-your job probably probably won't be getting any better.

It's Friday afternoon. You just can't write another line of code-but you're still stuck at the office...

What if every weekend could be a 3-day weekend?

18 May 2018 4:00am GMT