09 Apr 2018

feedPlanet Twisted

Itamar Turner-Trauring: Programming as natural ability, and the bandaid of long work hours

Your project deadline is getting closer and closer, and you're stuck: you don't know what to do. Your manager won't help, they just push you to work evenings and weekends-and when it looks like you're going to fail, they hand the project over to another programmer.

You've failed your manager, and there's a little voice in the back of your head telling you that maybe you're missing what it takes to be a programmer, maybe you just don't have the requisite natural ability.

That little voice is lying to you.

It's the other way around: your manager has failed you, and is compounding the failure by conveying a destructive mindset, what's known as a fixed mindset. To understand what I'm talking about, let's take a quick detour into the psychology of education, and then return to those long hours you've been working.

Growth mindset vs fixed mindset

Carol Dweck, a professor of psychology at Stanford University, contrasts two mindsets when learning:

According to Dweck's research, students with a growth mindset do better than students with a fixed mindset. A fixed mindset is self-defeating: it keeps you from learning, and it keeps you from even trying to learn.

In the context of programming this suggests that starting with the attitude that programming is a set of skills, skills that you can learn, will result in a much better learning experience.

But what does this have to do with long working hours?

You can't work smarter, so you gotta work longer

In an article that was one of the inspirations for this post, Carol Dweck points out that a common failure mode among educators is to praise effort, working harder, instead of praising learning. While they may claim to be encouraging a growth mindset, they are simply perpetuating a fixed mindset.

This failure mode appears in the software world as well. Let's assume for the moment that programming is a natural ability: just before we're born, the Angel of Software injects between 0 and 100 milliliters of Programming Juice into our soul. If you're really lucky, you might even get 110ml!

Now, given that each and every one of us only has a limited amount of Programming Juice, how can you maximize our output? You can't learn more, so there's no way to do things more efficiently. You can't improve your skills, so there's no way to become more productive.

So what's left?

All together now: WORK LONGER HOURS!

Working longer ain't smart

The truth, of course, is that there is no Angel of Software, there is no Programming Juice. Programming is just a bunch of skills. You can learn those skills, and so can most anyone else, given motivation, time, support, and some good teachers. And you can become more and more productive by continuing to learn.

If you believe in fixed talent, if you believe you can't improve, you won't try to learn. Long hours will be the only tool left to you.

When faced with a problem: just work longer hours.

When faced with another problem: work even longer.

You'll work and work and work, and you'll produce far less than you would have if you'd spent all that time improving your skills. And eventually you'll hit a problem you can't solve, and that you will never solve by working longer hours.

A growth mindset will serve you far better. You need to believe that skills can grow, and then you need to actually do the work to learn more and grow your skills.

And when you fail-and you will fail, because we all fail on occasion-take this as another opportunity to learn: look for the patterns and cues you should have have spotted. Having learned your lesson, next time you'll do better.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


09 Apr 2018 4:00am GMT

03 Apr 2018

feedPlanet Twisted

Moshe Zadka: Web Development for the 21st Century

(Thanks to Glyph Lefkowitz for some of the inspiration for this port, and to Mahmoud Hashemi for helpful comments and suggestions. All mistakes and issues that remain are mine alone.)

The Python REPL has always been touted as one of Python's greatest strengths. With Jupyter, Jupyter Lab in its latest incarnation, the REPL has been lifted into the 21st century. It has become the IDE of the future: interactive, great history and logging -- and best of all, you can use it right from your browser, regardless of your platform!

However, we still have 20th century practices for developing web applications. Indeed, the only development is that instead of "CTRL-c, up-arrow, return", we now have "development servers" which are not "production ready" support auto-reloading -- the equivalent of a robot doing "CTRL-c, up-arrow, return".

Using the REPL to develop web applications is pure bliss. Just like using it to develop more linear code: we write a function, test it ad-hocly, see the results, and tweak.

When we are sufficiently pleased, we can then edit the resulting notebook file into a Python module -- which we import from the next version of the notebook, in order to continue the incremental development. Is such a thing possible?

Let's start by initializing Twisted, since this has to happen early on.

from tornado.platform.twisted import install
reactor = install()

Whew! Can't forget that! Now with this out of the way, let's do the most boring part of every Python program: the imports.

from twisted.web import server
from twisted.internet import endpoints, defer
import klein
import treq

Now, let's start Klein app. There are several steps involved here.

root = klein.app.resource()

We take the Klein resource object...

site = server.Site(root)

...make a wrapping site...

ep = endpoints.serverFromString(reactor, "tcp:8000")

...create an endpoint...

ep.listen(site)
<Deferred at 0x7f54c5702080 current result: <<class 'twisted.internet.tcp.Port'> of <class 'twisted.web.server.Site'> on 8000>>

and like "Harry met Sally", eventually bring the two together for Klein to respond on port 8000. We have not written any application code, so Klein is currently "empty".

What does that mean?

async def test_slash():
    response = await treq.get('http://localhost:8000')
    content = await response.content()
    return content

This function uses Python 3's async/await features, to use treq (Twisted Requests) and return the result. We can use it as our ad-hoc debugger (but we could also use a web browser -- this is naturally hard to show in a post, though).

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c5532630>
b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">n<title>404 Not Found</title>n<h1>Not Found</h1>n<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>n'

Ah, yeah. Even / gives a 404 error -- we have literally defined nothing. OK, this is easy to fix:

@klein.route("/")
def something_useful(request):
    return 'Hello world'

Wait, did this work?

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c53d8d30>
b'Hello world'

Yep. But it's not a proper sentence...woops.

@klein.route("/")
def something_useful(request):
    return 'Hello, world!'

Nice. Punctuation. Force. Determination. Other nouns.

Did it change anything?

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c4b9e240>
b'Hello, world!'

Yes.

Incremental web application development. Without an "auto-loading" "not production grade" server.

We took advantage of several incidental issues. The Jupyter kernel is Tornado based. Twisted has both a production-grade web development framework, and the ability to run on top of the tornado loop. The combination is powerful.

03 Apr 2018 4:30am GMT

23 Mar 2018

feedPlanet Twisted

Itamar Turner-Trauring: You are not your tools

Do you think of yourself as a Python programmer, or a Ruby programmer? Are you a front-end programmer, a back-end programmer? Emacs, vim, Sublime, or Visual Studio? Linux or macOS?

If you think of yourself as a Python programmer, if you identify yourself as an Emacs user, if you know you're better than those vim-loving Ruby programmers: you're doing yourself a disservice. You're a worse programmer for it, and you're harming your career.

Why? Because you are not your tools, and your tools shouldn't define your skillset.

Your tools are not your skills

Ask a programmer to list their skills and more often than not you'll get a list of technologies. But technologies are just a small set of the skills you need as a programmer.

You need to gather requirements.

You need to debug code.

You need to design user experiences.

You need to build abstractions.

You need to avoid security problems.

And so on and so forth.

None of these skills are technologies. And if you think your only skills are technologies you won't notice the skills you don't have. And if you don't know what you're missing, you won't take the time to learn the skills that can make you truly productive and valuable.

Your tools are not your job

If you define yourself by your tools, you are limiting yourself to what jobs you can get.

First, because you won't apply to those jobs.

Second, because you will market yourself based on tools, instead of all the other skills that might get you that job anyway.

(I've written elsewhere about how you can get a job with technologies you don't know).

Your tools are not you

If your identity is tied up with your tools, you won't listen to people who use different technologies. Some tools are better than others at certain tasks. Some tools are interchangeable. But an expert using a bad tool can often do more than a novice with a bad tool.

Spending your time fighting over which tool is better is a waste of your time. Instead, spend your time listening and learning from everyone, whatever tools they use: most skills will transfer just fine.

The technologies you use, the tools you build with, are just that: tools. Learn to use them, and learn to use them well. But always remember that those tools are there to serve you, you are not there to serve your tools.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


23 Mar 2018 4:00am GMT

20 Mar 2018

feedPlanet Twisted

Moshe Zadka: Running Modules

(Thanks to Paul Ganssle for his suggestions and improvements. All mistakes that remain are mine.)

When exposing a Python program as a command-line application, there are several ways to get the Python code to run. The oldest way, and the one people usually learn in tutorials, is to run python some_file.py.

If the file is intended to be usable as both a module and as a command-line parameter, it will often have

if __name__ == '__main__':
    actually_run_main()

or similar code.

This sometimes has surprising corner-case behavior, but even worse -- some_file.py is not looked for in either $PATH or sys.path, it must be explicitly handed. It also changes the default Python path from including the current directory, to including the location of some_file.py.

The new recommended modern way, of course, is to set entry_points in the setup.py file. When the distribution is installed, a console script is auto-generated and added to the same place the Python interpreter is found. This means that we need to think carefully about the other things that might have the same name on our $PATH to avoid collisions.

There is a third way, which is subtle. When Python sees the -m <some name> option, it will look for a module or a package by that name. If it finds a module, it will run it with __name__ being "__main__" in order to trigger the path that actually does something -- again leading to some, if not all, issues discussed earlier.

However if it finds a package it will run its __main__.py module (still setting __name__ to "__main__") -- not its __init__.py.

This means that at the top of __main__.py we can invert the usual logic:

if __name__ != '__main__':
    raise ImportError("Module intended to be main, not imported",
                      __name__)

from . import main_function
main_function()

This allows running python -m <some package>, but anyone who tried to accidentally import <some package>.__main__ will get an error -- as well they should!

Among other things, this means we only care about our sys.path, not our $PATH. For example, this will work the same whether the package is installed to the global Python or --user installed.

Finally, if an entrypoint is desired, one can easily be made to run __main__:

entrypoint = toolz.compose(lambda _dummy: None,
    functools.partial(runpy.run_module,
                      "<some package>",
                      run_name='__main__'))

Using the builtin module runpy, the builtin module functools and the third party module toolz.

20 Mar 2018 3:30am GMT

14 Mar 2018

feedPlanet Twisted

Moshe Zadka: Random Bites of Pi(e)

In today's edition of Pi day post, we will imagine we have a pie. (If you lack imagination, go out and get a pie.) (Even if you do not lack imagination, go out and get a pie.)

As is traditional, we got a round pie. Since pies are important, we will base our unit of measure on this pie -- the diameter of the pie will be 1.

Since we had to carry it home, we put it in a box. We are all ecologically minded, of course, so we put it in a box which is square -- with its length size 1.

We note something interesting -- the box's bottom's area is 1x1 -- or 1. The radius of the pie is 1/2, so its area is pi * 0.25.

As we are driving home, the pie on our passenger seat, we start wondering how we can estimate Pi. Luckily, we got some sugar. What if we sprinkled some sugar, and took notes for each grain, whether it was on the pie, or not?

Let's use Python to simulate:

import random
import attr

First, we need some randomness generator. Then, we also want to use the attrs library, because it makes everything more fun.

We make a Point class. Other than the basics, we give it a class method -- a named constructor which will generate a random point on the unit square -- this is where our sugar grain falls.

We also give it a way to calculate distance from another point, using the Pythagorean theorem.

@attr.s(frozen=True)
class Point:
    x = attr.ib()
    y = attr.ib()

    def distance(self, pt):
        return ((self.x - pt.x) ** 2 + (self.y - pt.y) ** 2) ** 0.5

    @classmethod
    def unit_square_random(cls):
        return cls(x=random.random(), y=random.random())

The center of the pie is at 0.5 by 0.5.

center = Point(0.5, 0.5)

A point is inside the pie if it is less than 0.5 away from the center.

def is_in_circle(pt):
    return center.distance(pt) < 0.5

Now we are ready. Even with just 100,000 grains of sugar, we get 2-digit accuracy.

inside = total = 0
for _ in range(10 ** 5):
    total += 1
    inside += int(is_in_circle(Point.unit_square_random()))
print((inside / total) * 4)
3.14052

14 Mar 2018 3:00pm GMT

09 Mar 2018

feedPlanet Twisted

Itamar Turner-Trauring: Why you're losing the battle for high-quality software

Every day you go to work and you have to fight. Your boss wants to deliver features as fast as possible, and so you have to either argue for best practice, or work extra hours to do things the right way.

You need to fight for readable and maintainable code. And you need to fight for reusable code. And you need to fight for tests.

And you fight and you fight and you fight, and you keep on losing.

It doesn't have to be this way. Software development doesn't have to be a fight. Strange as it may seem, in this game the only winning move is not to play.

Instead of playing "Let's Write High-Quality Software", there's a different and better game you can play: "Let's Solve The Problem".

Avoid false dichotomies

The problem with the game of High-Quality Software is that it's based on a dichotomy, with only two options: low-quality and high-quality. I started this article with another dichotomy: arguing with your boss vs. working longer. And a lot of technical discussions quickly devolves into dichotomies, e.g.:

Dichotomies are tempting, and perhaps even built-in to the way we understand the world: there's a left hand and a right hand, a wrong way and a right way. But there's nothing inherently superior in just having two choices. In fact, all it's doing is making you a worse engineer, because you're focusing on arguing instead of focusing on solving the problem.

To combat this tendency, Gerald Weinberg suggests the Rule of Three: always consider at least three solutions to any problem. Let's start with our first dichotomy: arguing with your boss vs. working longer hours to do things the right way. If there's a third choice, what might it be?

Stop arguing, start listening

When your boss or colleagues argue for a specific design, instead of telling them why they're wrong, listen to their reasons. Behind every design proposal is a set of goals, motivations, presumed constraints: those are the things you need to address. If you criticize their proposal based on goals they don't care about, you're not going to get anywhere.

So first, listen.

Once you understand why they want what they want:

  1. Consider more than the initial two choices, their proposal and your initial reaction.
  2. Try to find a solution that addresses both your goals.
  3. Explain you thinking in ways that are relevant to their goals, not just yours.

Example scenario: testing

Your boss proposes writing code without unit tests. Why? Because they want customers to be happy. Since customers have been complaining about how long it takes for new features to be delivered, your boss believes this is one way to speed up delivery.

You want to write unit tests. Why? You want code to be maintainable over time.

Merely arguing for unit tests isn't going to address your boss' concerns. But if you look past the initial false dichotomy, there are other solutions to consider. For example:

No doubt there are many more potential solutions to the standoff.

There's always another solution

There is almost never a single correct solution to any problem, nor a single best solution. You can solve problems by relaxing unnecessary constraints, by focusing on different levels of the situation (organization, process, code), by redefining the problem, and more. Worst comes to worst, you can address many problems by switching jobs; different people like different environments, after all.

So don't try to win technical arguments, and in fact, don't treat them as arguments at all. When you disagree with someone, take the time to listen, and then try to come up with a solution that address their concerns and yours. And if your first idea doesn't do that… it's time to come up with a third, and fourth, and fifth solution.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


09 Mar 2018 5:00am GMT

04 Mar 2018

feedPlanet Twisted

Itamar Turner-Trauring: Only code at work? That doesn't make you a worse programmer

At the end of the day you're done with work, you go home-and you don't spend any of your free time coding. And that's fine, you have other things going on in your life. But your coworker does spend another 20 hours a week coding, and all that practice means they'll end up better programmers than you, and so they'll get promoted faster, and they'll get paid more. And that's not fine.

It's also not true.

That inadequacy you're feeling is based on false assumptions, a misunderstanding of the skills it takes to be a good programmer. Enthusiasts who love coding as a hobby are valuable, yes, but so are you: even if extra hours of coding result in better skills-often a very doubtful assumption-their approach is not inherently better.

All but the smallest of programming projects are team efforts, requiring a variety of skills, knowledge, and mindsets. The mindset of someone who view programming as a tool to do their jobs-let's call them pragmatists-is just as useful as the mindset of enthusiasts.

To understand why, let's go through the lifecycle of the enthusiast programmer, see how their skills evolve, and compare them to the pragmatists' approach. As you'll see, enthusiasts and pragmatists start out with different strengths, and software teams can benefit from both.

The enthusiast's path

The enthusiast's starting point is a love of programming: they love to code. They take joy in writing software, and understanding the intricacies of the technologies they use.

Over time, as they become more skilled and experienced, they expand their point of view. Instead of just wanting to write code, they want to write good code, high-quality code: they want to be proud of their work. So now they focus not just on the joy of coding, but also on less enjoyable but important disciplines like testing.

If they continue to grow in skill and experience, they will eventually hit a point where they see code as a tool: a means to an end. Software is still fun, yes, but it also needs to be written to achieve certain goals. At this point quality becomes less of an obvious objective criteria: every situation is different, and what works in one situation doesn't work in another.

Strengths and weaknesses: Enthusiasts often have a strong grasp of the tools and technologies they use. On the other hand, they are sometimes prone to holy wars and zealotry (they feel it's important to choose the very best tool), and until they finally realize software is a means, not an end, they have a harder time working towards goals that aren't about writing software.

The pragmatist's path

The enthusiast's weaknesses are the pragmatist's strength. Pragmatists start out seeing software as a means to end: they will tell you "I like writing software, but mostly I care about what the software lets me do." So what can take the enthusiast years to learn-a focus on goals-comes naturally to the pragmatist.

Of course, if they're not deliberately focusing on learning, pragmatist can sometimes suffer from less understanding of tools and techniques. This can be mitigated by learning on the job, and making sure to build awareness of available technologies.

Software is not a competition

With time and practice, the strengths of enthusiasts and pragmatists can converge. Even so, different people will typically end up with different skillsets, different approaches, and different ways of building software.

This is how it should be. Software projects are too big to be built by one person, too complex to be encompassed by one view point.

When you compare yourself to your peers, you shouldn't ignore your weakness, but you should also value your own strengths. You can and should learn from your colleagues, but chances are there's something you can teach them.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


04 Mar 2018 5:00am GMT

15 Feb 2018

feedPlanet Twisted

Itamar Turner-Trauring: I took a sick day today, and that's OK

I skipped work today because I'm sick: nothing terrible, nothing life threatening, I'm just-exhausted. I definitely can't focus on code, I can't even focus on television; so far I've been re-reading a favorite old novel, and looking at Twitter when I get distracted.

The world will not come to an end because I'm not working. My company won't collapse because I took the day off. My project will not fail.

I'm sick and I'm taking the day off, and that's OK. And if you can't take a day off-whether because you're sick, or just because you feel like it-then something is very very wrong.

And now… I think I'm going to lie down on the sofa.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


15 Feb 2018 5:00am GMT

11 Feb 2018

feedPlanet Twisted

Itamar Turner-Trauring: The futile comfort of working long hours

Working long hours as a programmer can be both comforting and comfortable. After all, you have an easy solution to every problem: a few more hours at your desk, a few more hours staring at a screen in the dark. And working long hours is a natural attitude to take, since at the root of it, working long hours comes from valuing work, your craft as a programmer, and your obligations as a professional.

There is a different approach you can take. Instead of valuing work, you can value outcomes: outcomes at your job, outcomes in your life, outcomes in the world. Valuing outcomes is not comfortable. It forces you to reconsider and doubt your every step, and if you fail-and we all fail occasionally-your hard work becomes worthless.

But while valuing outcomes is uncomfortable and difficult, if you do it long enough you will find yourself-every once in the while-with the ability to grasp the moment and change the world. Just a little, in whatever corner you live in, but change it nonetheless.

The most direct way to learn this attitude is to reduce your working hours. If you had to do your work in 32 hours a week, instead of 45, or 50, or 60-what would need to change?

The experience of working fewer hours

Testing and quality

If you work long hours, you can pick some level of quality and stick to it:

But if you work fewer hours, you will need to make decisions and tradeoffs, because you won't have those extra hours. Some code will need automated tests for stability and maintainability, some code will need manual tests every time you change it, some code will be used once and thrown away. And you will need to make that choice, every single time. Which is to say, you will be forced to think about why you're writing this code, and what it will be used for.

Planning

If you work long hours, the direction you go in matters less: if it's the wrong direction, you can just work a few more hours when you eventually find out after you finish your task.

But if you work fewer hours, you will find yourself constantly plagued by doubt: is this the right direction? Are you doing the right thing? You may end up changing direction half-way, abandoning your work-however much it pains you-when you realize there's a better way to your destination. You will have to spend much more time thinking and planning up front, to make sure you arrive at the right place on time.

Speaking of time, deadlines cause less worry when you work long hours. If you hit your deadline, great. If you didn't, well, you worked long and hard, and who can blame you if you failed?

If you work fewer hours, deadlines are a constant gnawing presence, always getting closer. You will need to plan for them well in advance, always asking questions, on the lookout for unstated requirements and forgotten deliverables. And sometimes you'll be forced to find new solutions so you can hit those deadlines in time-and, of course, sometimes you will fail to find those solutions. And then you'll look bad, because you didn't work on the weekend, so whose fault is it the deadline was missed?

Your value

When you work long hours, it's easier to demonstrate how valuable you are to your boss: you were there on the weekend, after all. If you work fewer hours, well, who can say you're really getting much done? Did you deliver anything of value? How will you prove it?

Even worse than your boss is your own self-image. When you work long hours, your self-worth is about your work: you work hard, and that at least is something to be proud of, even if the project failed. If you work fewer hours you won't be able to feel proud of all the hours you worked. You'll be forced to consider the results, the outcomes: if you failed to deliver, you failed.

From comfort to effectiveness

Go through all this and- you'll become effective.

Valuing outcomes is not comfortable, but it is empowering. And good day or bad, you get to go home at a reasonable hour and do whatever you feel like. Which feels pretty damn good too.

You don't have to take my word for it, either: it's easy to try out. If you're working long hours, try spending a month working 40 hours a week, no more. Once you can no longer rely on working longer hours to solve all your problems, you'll soon find yourself approaching your work very differently.

And if you want a head start on learning these skills, read my book, The Programmer's Guide to a Sane Workweek.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


11 Feb 2018 5:00am GMT

02 Feb 2018

feedPlanet Twisted

Moshe Zadka: The Python Toolbox

I have written before about Python tooling. However, as all software, things have changed -- and I wanted to write a new post, with my current understanding of best practices.

Testing

As of now, pytest has achieved official victory. Unless there are overwhelming reasons to use something else, strongly consider using it as your test runner. The only good reason is an existing project, with an existing test runner -- and even there, checking if pytest will just run your tests as is is worthwhile.

Even when using Twisted, unless you are implementing a new reactor, tests should be using in-memory reactors -- so the usefulness of trial is limited.

Static checking

In the static checking arena, flake8 and pylint are both still going strong. There are less and less reasons not to use both, especially if starting a project from scratch.

The flake8 plugin ecosystem is rich, and it is useful to look around and see if useful things are there. For example, the ebb-lint plugin is extremely strict about coding conventions -- hard to introduce to existing code bases, but wonderful when starting out a new one.

In the meantime, pylint has decent static code flow analysis which can often prevent bugs. Note, however, that Python static code analysis is hard, and pylint is not perfect. For example, it will often strugle with attrs-based classes.

Last but not least, mypy has made a showing in the field. It supports both Python 2 an 3, and allows annotating functions with types, in order to find mismatches before running the code. This is especially useful at API "boundaries", where the unit tests tend to cross less.

Test metarunners

The tox testing system is still the golden standard in Python. It can test complicated dependency matrixes, and allows configuring test commands flexibly. Its documentation is somewhat lacking, sadly -- it is often the case new tricks are only apparently after reading someone else's tox file.

Output

Building wheels, especially if the project has no native-code extensions, is nowadays considered standard. The best place to build wheels is in tox, when configuring a test that will build a wheel, install it, and then test against the installed wheel.

The best and most reliable way to upload wheels, and source distributions, to PyPI is to use twine. It used to be a good idea to test against the test PyPI server, but nowadays it is best to set up a devpi server for local testing.

When building applications, pex is a great output format. It allows a one-file install.

Future

The future is bright -- pip 10 is slated to come out, supporting the pyproject.toml format -- and hopefully the next post in the series will explain how to make packages using flit, with no setup.py.

02 Feb 2018 6:20am GMT

01 Feb 2018

feedPlanet Twisted

Itamar Turner-Trauring: Learn these programming skills before your inevitable death!

There is so much to learn as a programmer it's hard to even know where to begin. And even if you do begin, there's always something newer and shinier to distract you, and so you end up making these giant lists of things to learn and they get longer and longer and longer and you feel like shit about it but really you don't have the time to learn all of it, now do you?

So, why not take a few minutes to stop worrying about the things you should be learning, and instead let's talk a little about all the things you don't need to learn. And maybe when we're done you'll feel a little better.

But first, let's talk about-

DEATH.

We're all gonna die (eventually)

Someday you're going to die. So will I. So will everyone else, our friends and enemies both.

Typically one measures one's lifespan in years or days, but I once went and measured it in books. I did the math and figured out how many books I could read before I reached my presumptive death from old age. At the time I was reading about two books a week, but even so the number didn't seem anywhere near sufficient.

And having started down this morbid path, I arrived at an even worse place. Every time I read a book I'd get to thinking: "Is this book worth reading before I die? Shouldn't I be reading something more edifying than this entertaining yet trashy novel? Isn't re-reading a book a complete waste of my time?" Instead of enjoying the book I was reading, I was worrying about some other better book I could have been reading instead.

Eventually I got over it. I won't be able to read every book I want to before I die. Neither will you.

Sorry.

It's not so bad, though. When you're lying there dying you'll probably be thinking about the friends and family you'll miss. Or maybe you'll just be tired, and looking forward to an end to your pain and sorrow. Or, if you're having a really bad day, you'll be thinking that you shouldn't have had that last drink before driving home-don't drink and drive, kids. Better yet, don't drive at all; commuting is a terrible way to spend your life.

In any case, when it's time for you to die you probably won't be worrying about the books you haven't read. And as you lie on your deathbed, looking back at your life, you definitely won't be worrying about that new JavaScript framework you didn't get to play with.

Some skills you don't need to learn

There are many skills I do think you should learn (I have a whole pile of posts on this here website, in fact), but honestly-it's just software. If you don't learn these skills before you die, that's really not a big deal.

Software is a tool: tools are useful, and important, and you need them to build many things. But our tools are there to serve us, we should not be serving our tools.

Not to mention all the skills you really don't need to learn.

You don't need to learn every blindingly shiny new technology that will End Poverty and Bring Peace to Humanity. It probably won't do either, and quite possibly it won't turn out to be good for much at all. I started my career programming building multimedia CD-ROMs, which were really hot for about 6 months in the mid '90s, and somewhere in a landfill there's still boxes of old unusable CDs I worked on that no one cares about.

You don't need to learn everything programming language, certainly not in your spare time. You can and should learn those on the job, as and when they become useful.

You don't need to learn how to use every new library, tool, or framework. Just knowing they exist is usually more than enough: when and if you need them, you'll know they exist and go learn them then.

Some things to do before you die that are more important than learning another programming skill

Spend more time with your friends.

Spend more time with your family (unless you don't get along, sorry).

Eat good food.

Visit UNESCO World Heritage sites.

If you haven't seen one, a full solar eclipse is amazing.

Make the world a better place, even if it's just a little.

Whatever you think is important and worthwhile.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


01 Feb 2018 5:00am GMT

30 Jan 2018

feedPlanet Twisted

Itamar Turner-Trauring: Not an expert? You can still teach

So your manager stops by and says "hey, we've got some new employees starting in a month, and you know SomeTechnology best, I'd like you to get them up to speed." And you nod and smile and feel the impostor syndrome kicking in: you don't know how to do this.

Software is complicated. Sometimes even when you've been using a tool or a language for a year or two, you might only know a small part of the functionality. It's already tough and intimidating to teach as it is, it's even worse when you're not confident in what you know. How can you teach when you're not an expert?

But as it turns out, being an expert can impede your ability to teach. You can become a better teacher by taking advantage your lack of expertise.

The problem with being an expert

Experts can actually make worse teachers. Specifically, experts often suffer from "expert blind spot", where their own knowledge blinds them to the way their students are thinking (I first learned of this concept from the book How Learning Works).

Experts differ from beginners in a number of ways that impact learning.

  1. Experts have much denser and interconnected conceptual models. Things that are obviously related to them are not at all obvious to beginners, who have sparsely connected conceptual models.
  2. Experts will skip steps, doing certain things so automatically they don't even realize they're doing them. From the beginners' point of view this results in unexplained jumps.
  3. Expertise is often unconscious. When asked, experts may not be able to explain why they're doing what they're doing, even if they know it's the right thing.

As a result, being an intermediate learner, where you know how to do a task but still need to consciously walk through the steps, can actually make you a better teacher.

Teaching when you're not an expert

Since you're not an expert, you don't need to pretend to be one. Instead you can teach the valuable knowledge you do have: the combination of the understanding of the technology, and the understanding or recent memory of how beginners think.

By showing how to deal with mistakes, and by admitting mistakes to your students, you're also making the topic less intimidating. Unlike some experts, who get confused and perhaps annoyed when mistakes happen-they never make those mistakes after all-you are giving your students the confidence that they too can work through problems. (Thanks to reader Jake for this point.)

Go teach!

Teaching doesn't need to be scary. It can just be you sitting down at a computer and saying "this is tricky, but I've figured out some ways to get things done, and so can you. Let me show you how."

This also applies to teaching more broadly. Every once in a while you'll see a conference call for proposals, and you'll say to yourself "I'm not an expert, I can't teach anything." But the truth is you can: your perspective as an intermediate learner is often much more valuable to beginners than that of an expert. Submit a talk proposal even if you're not an expert: you too have a valuable perspective.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


30 Jan 2018 5:00am GMT

23 Jan 2018

feedPlanet Twisted

Itamar Turner-Trauring: How to get a job with a technology stack you don't know

Have you ever read an amazing-sounding job posting, and then felt that sinking feeling in your stomach when you reached the list of required technologies? Maybe you're a Python programmer, and they use Ruby on Rails; maybe you're mostly a front-end developer, and they want back-end skills. Whatever the missing technologies are, this is an awesome job you don't have a chance of getting.

Or do you?

In fact, it is possible, albeit not always easy. Years ago I got a job writing C++, when all I knew was Python and Java. And for the past few months I've been doing scientific computing for the first time, with a new toolchain, and math skills I haven't used in almost 20 years.

In this post I'll cover:

Apply anyway

Even if you feel you're not fully qualified for a job, you should still apply:

First, the list of technologies may be irrelevant. Sometimes the technologies listed are things the company might want to use someday, not what they actually use today. Sometimes they are perfectly happy hiring candidates with different technologies, and they put the list of technologies down because they aren't very thoughtful about how they write job ads.

But what if they do actually want those technologies? The list of technologies and skills is what the company would like in the ideal world, not necessary what they can get in practice. In the ideal world many companies would probably also like to pay you half as much and have you be twice as experienced, with the ability to create gold out of lead and summon unicorns at will. In practice, companies hire the candidates they can get, not the magical and often non-existent candidates they dream of.

Finally, technologies aren't everything. There are many other skills you have as a programmer, and some of them may trump the particular technologies you lack. We'll revisit this in the section on pitching yourself.

Which companies to apply to

It's hard to say as an outsider which companies will be more flexible, but here are some things to look for:

How to pitch yourself

Once you've picked the companies to apply to, you want to customize your cover letter, resume, and if you get there your presentation at the job interview. Here are some ways to do that:

The key is to research the company, try to understand their needs, and then demonstrate you can have value to them beyond just the list of technologies you know.

Next time you're looking for a job, don't limit yourself only to jobs with a technology stack you know. Look for jobs that excite you, for jobs you think you can do with just a little ramp-up time. And then apply for those jobs, knowing that most of them will ignore you-but it only takes one yes to get an exciting new job, and an exciting new opportunity to learn new skills.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


23 Jan 2018 5:00am GMT

18 Jan 2018

feedPlanet Twisted

Itamar Turner-Trauring: The worst kind of resume is a list of technologies

Are you having a hard time getting responses to your resume? Do you know you have the skills for a job, but have a hard time convincing the company to hire you?

If your resume starts out with a long list of technologies you are selling yourself short. Knowing and using programming languages, libraries, and so on is a critical job skill for a software developer, but it's only part of what makes a good programmer. And the skills you aren't mentioning may well be keeping you from getting hired, even if you already have them.

In this post I'll discuss:

The most important part of your resume

Recruiters spend very little time reading resumes, and more broadly people tend to skim when reading. That means the first thing a reader see when they read your resume will have the most impact. The lower on the page, the less likely it is to be reached.

You should imagine the hiring manager as looking for some hint or phrase that will make them think "aha, this sounds promising, I should keep reading." And you should also imagine them giving up looking after less than 10 seconds.

That means you need put the most important pieces of information at the very top of your resume. The first sentence, the first paragraph, no lower. Your goal isn't to convey everything, you just want to make yourself sound interesting and relevant enough that the recruiter keeps reading.

For example, let's say you worked in bioinformatics two jobs ago, and you're applying to a bioinformatics job again. Your opening paragraph should mention your biotechnology work; that way the reader will be motivated to keep reading. Having given that motivation, you can still do the normal reverse chronological job history in the second part of your resume-but perhaps the paragraph for your current job should be a little shorter if it's less relevant.

Don't lead with a list of technologies

Since you need to make yourself sound interesting in the first few seconds, a list of technologies is a bad way to start. Merely claiming to know a technology doesn't tell the hiring manager how useful of an employee you'll be.

In some jobs knowing particular technologies will give you a head start, which is why some recruiters foolishly use technology keywords as a filter, but it's never really enough to do your job as a programmer. So spending precious moments of attention listing the technologies you know is a bad way to convey your value:

What to do instead

Instead of opening with a list of technologies, open with a paragraph demonstrating the ways in which you're a valuable employee, ideally tailored to the company you're applying for. Specifically, you want to convince the hiring manager you can do the job they're hiring for, which is either solving problems, or at more experienced levels identifying and solving problems. Writing software is merely a means to that end.

So you want a starting paragraph that emphasizes:

And you want to include just enough real-world examples that you will convince them to keep reading. Once you've got them reading the main job-listing section of your resume you can go into much more detail.

You can still have a list of technologies if you want, but don't waste the precious first half page of your resume on it. Personally I don't bother: I just mention names of relevant technologies at the jobs where I used them.

If you're currently looking for a job, go look at your resume, set a timer for 10 seconds and read it from the start. What will the recruiter learn about you? And is that all you want them to know? If all they've learned is that you know tool X or language Y, it's time to make some changes.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


18 Jan 2018 5:00am GMT

08 Jan 2018

feedPlanet Twisted

Itamar Turner-Trauring: How to become a part-time programmer: an interview with an expert

For some people, a 40-hour workweek is something to aspire to; for others, it's still too much time taken up by a job. If you fall into that second category, if you want more time for hobbies, family and friends, or working on your own software projects, you too might dream of working less than full time.

But how do you get there? Almost no one advertises part-time programming jobs-believe me, I've me looked.

The answer: negotiation. I've negotiated a shorter workweek a few times myself, and I've met other programmers who have done so as well, some with just a few years of experience. And of all the programmers I've met who've negotiated part-time work, Mike's record is the most impressive.

Mike has spent pretty much all his career working part-time: he's been working part-time for more than 15 years. To help you get to a shorter, saner workweek, I sat down to interview Mike about how he does it.

Q. What does a sane workweek mean to you?

MIKE: Well, I only ever worked full time for about 1 year, and I learned pretty quickly that a sane workweek for me was less than 40 hours a week. I guess it's up to each individual, but I think most of us are forced to work more than we'd like, and at least for me 30-32 hours is better.

I want to work on average less, but I make it clear [when starting a job] that [I understand] things happen, stuff needs to get done. I would definitely work longer hours here and there.

Q. How did you realize you wanted to work less hours?

MIKE: Part of the reason I decided to demand this so early was, I started that first job when I was in university, and I stayed on for several years part-time while I was school. My contract was 10 hours a week, not very much but I was also going to classes and had to do homework and shit. I suppose that got me used to being able to go cycling or climbing or hiking on fairly short notice.

They were terrible at planning at this company, when a contract deadline was coming up everyone was working 60 to 70-hour weeks, having dinner at the office all the time. I wasn't forced to get caught up in that, since I had my excuse of going to school. I worked a lot more than 10 hours I was contracted to, though.

I saw this cycle there a lot, where managers would pull numbers out of their ass and promise them to clients, and then all the programmers were panicking for the last month. The panic mode of getting this done. [My feeling is] I'm not committed to those numbers you pulled out of your ass to tell your client, so partly it was reaction to that, seeing these people spending their entire lives at the office. I wanted to carve out my time early on.

Q. So how did you end up part-time after that first year?

MIKE: At that I job I tried to quit.

But they basically offered me more money, so I was like, "what if you gave me 75% of that, and I worked less?" And I guess they wanted to keep me around and they went for that.

At that job I was on payroll as a full time employee, but I got a bunch more holidays. So I got a quarter of my time in holidays; 72 days off a year? I was constantly booking time off, which was interesting.

That was how I landed my first part time thing. And then I did eventually quit that company and worked elsewhere, but I had found a new job while I was still in a part time situation, so it was a lot easier to demand a similar deal going forward.

Q. How many companies have you done this with? How many part time jobs?

MIKE: A bunch. All of my jobs. 7, no, 8.

(Mike goes off to look at his resume)

If you count the two I have now, two part time contracts, then 9.

Q. How exactly did you negotiate for shorter hours at later jobs?

I have several interviews here and there. And sometimes-if it's a company where I'm meh, not that serious about or interested in-I'll send them an email early in the process saying "hey I want to work part time." And then if they won't go for it I won't bother.

If I definitely wanted to work there, I'd go for the interview without telling them [I wanted a shorter workweek]. And if I got back for subsequent interviews or I got an offer I'd say "now I'm working 75%, I want to work part time somehow, I'm open to various different arrangements, I'm interested in working with you, lets work something out." Mostly if the companies were in the job offer stage they'd consider it very seriously. One was "no, no way," they didn't want to talk to me anymore. Mostly they're interested in negotiating somehow.

Q. What kinds of counter-offers did you get?

MIKE: Sometimes it was stuff like "I want to hire you, how do I sell this management? Help me sell it to management." That job took me 6 months to get from interview to legitimate part time job offer. That was the longest, and also unbeknownst to that company I'd been laid off: I'd started negotiating when I had a job, and was subsequently laid off. They wanted me to sell it to management. I said, "I don't want to work 5 days, how much happens Fridays? Not much? Then I'll work Monday to Thursday."

Usually I negotiated the full-time salary first, and then the details of pro-rating later: mostly it was about the details, once you'd convinced someone you were worth having. The easiest ones to negotiate are the ones which are 80% time, especially one day a week off. Usually I'd say Monday or Friday, and get a longer weekend, go to the mountains. But one of the people I inspired got himself an 80% contract, and he said "you've got to take Wednesday off." With Wednesdays, you only ever work two days in a row. It was awesome.

I always set expectations: "you're getting the best 4/5ths of my time, but only 4/5ths. I'm not going to pound out as much code." Realistically you kinda do, but the expectation should be if you're not there Wednesdays you shouldn't be doing as much work. That's part of the argument, that you're doing almost as much useful work.

Q. Did you find asking for a shorter workweek got easier over time?

Yeah, for sure, it definitely got easier. After the first couple times I felt more confident asking in the first place. It got easier in that sense.

Q. When you tell people you don't work full time, how do they react?

It's still considered weird by a lot of my friends, but I also get the other reaction, "I wish I could do that." Some people when you start new jobs, once you'd explained why you weren't there Wednesday, they'd say "I totally want that" but then I'd say I only get paid 80%. Once they'd realize they'd get paid less, [they'd say] "I'm not doing that."

Other people would say "cool, I want that." Definitely people I've worked with over the years have been inspired to do that, which is pretty nice.

Q. Why was the lower salary not an issue for you?

MIKE: I don't know, I guess I decided early on free time was more important than more money. Programmers are paid pretty well in my experience, 80% of a programmer job is plenty of money to get by on. Typically it's a lot more than what a lot of people are getting.

For me it was putting a premium on my time, and wanting to have that time to do other things. I've got hobbies, always had hobbies, different sports over the years which takes time. But even just stuff like having time to work on your own programming project, which I think is really valuable. Valuable enough to me that I'll actually pay money for it in the form of taking less salary.

I definitely don't have any regrets for doing that.

Q. Any final advice?

MIKE: My biggest piece of advice is that you have to be willing to quit your job to get a sane workweek, or you won't sound convincing when you ask for it. If you're looking for a new job, just ask, and if you're still at a job, then you're in a better position to ask.

Just go for it!

Itamar here: while I agree that working less than full time is wonderful, it also isn't for everyone. Perhaps to you a sane workweek means working for yourself, or working remotely, or just getting a job where you can go home at 5PM.

But whatever a sane workweek means to you, the basics are the same. As in Mike's case, you need the skills to be productive and valuable, you need to be able to negotiate, and you need enough financial security to have a strong negotiating position.



You shouldn't have to work evenings or weekends to succeed as a software engineer. Take control of your time and your career by reading The Programmer's Guide to a Sane Workweek.


08 Jan 2018 5:00am GMT

07 Jan 2018

feedPlanet Twisted

Glyph Lefkowitz: Tips And Tricks for Shipping a PyGame App on the Mac

I've written and spoken at some length about shipping software in the abstract. Sometimes I've even had the occasional concrete tidbit, but that advice wasn't really complete.

In honor of Eevee's delightful Games Made Quick???, I'd like to help you package your games even quicker than you made them.

Who is this for?

About ten years ago I made a prototype of a little PyGame thing which I wanted to share with a few friends. Building said prototype was quick and fun, and very different from the usual sort of work I do. But then, the project got just big enough that I started to wonder if it would be possible to share the result, and thus began the long winter of my discontent with packaging tools.

I might be the only one, but... I don't think so. The history of PyWeek, for example, looks to be a history of games distributed as Github repositories, or, at best, apps which don't launch. It seems like people who participate in game jams with Unity push a button and publish their games to Steam; people who participate in game jams with Python wander away once the build toolchain defeats them.

So: perhaps you're also a Python programmer, and you've built something with PyGame, and you want to put it on your website so your friends can download it. Perhaps many or most of your friends and family are Mac users. Perhaps you tried to make a thing with py2app once, and got nothing but inscrutable tracebacks or corrupt app bundles for your trouble.

If so, read on and enjoy.

What changed?

If things didn't work for me when I first tried to do this, what's different now?

There are still weird little corner cases you have to work around - hence this post - but mostly this is the story of how years of effort by the Python packaging community have resulted in tools that are pretty close to working out of the box now.

Step 0: Development Setup

Get a good Python. Use Homebrew, and brew install python3. If you need python 2, brew install python2. Don't use the System python. Probably nothing will work.

You probably also want to use a virtualenv for development. This post is about how to build a for-real thing that other people can download, but part of the magic of Python is the interactive, real-time dynamic nature of everything. Running the full build pipeline every time you change a file or an asset is slow and annoying. However, there's a weird thing where certain parts of the macOS GUI won't work right (in PyGame's case, mostly keyboard focus) unless your code appears to be in an application bundle.

I made this dumb little thing which lets you fake out enough of this that the OS won't hassle you: you just need to pip install venvdotapp; venvdotapp inside the virtualenv where you're making your pygame app.

Finally: pip install all your requirements into your virtualenv, including PyGame itself.

Step 1: Make an icon

All good apps need an icon, right?

When I was young, you just popped over into ResEdit Resorcerer MPW CodeWarrior Project Builder Icon Composer Xcode and created a new ICON resource cicn resource .tiff file .icns file. Nowadays there's some weird opaque stuff with xcassets files and Contents.json and "Copy Bundle Resources" in the default Swift and Objective C project templates and honestly I can't be bothered to keep track of what's going on with this nonsense any more.

Luckily the OS ships with the macOS-specific "scriptable image processing system", which can helpfully convert an icon for you. Make yourself a 512x512 PNG file in your favorite image editor (with an alpha channel!) that you want to use as your icon, then run it something like this:

1
$ sips -s format icns Icon.png --out Icon.icns

somewhere in your build process, to produce an icon in the appropriate format.

There's also one additional wrinkle with PyGame: once you've launched the game, PyGame helpfully assigns the cute, but ugly, default PyGame icon to your running process. To avoid this, you'll need these two lines somewhere in your initialization code, somewhere before pygame.display.init (or, for that matter, pygame.display.<anything>):

1
2
from pygame.sdlmain_osx import InstallNSApplication
InstallNSApplication()

Obviously this is pretty Mac-specific so you probably want this under some kind of platform-detection conditional, perhaps this one.

Step 2: Just Include All The Dang Files, I Don't Care About Performance

Unfortunately py2app still tries really hard to jam all your code into a .zip file, which breaks the world in various hilarious ways. Your app will probably have some resources you want to load, as will PyGame itself.

Supposedly, packages=["your_package"] in your setup.py should address this, and it comes with a "pygame" recipe, but neither of these things worked for me. Instead, I convinced py2app to just splat out all the files by using the not-quite-public "recipe" plugin API:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
import py2app.recipes
import py2app.build_app

from setuptools import find_packages, setup

pkgs = find_packages(".")

class recipe_plugin(object):
    @staticmethod
    def check(py2app_cmd, modulegraph):
        local_packages = pkgs[:]
        local_packages += ['pygame']
        return {
            "packages": local_packages,
        }

py2app.recipes.my_recipe = recipe_plugin

APP = ['my_main_file.py']
DATA_FILES = []
OPTIONS = {}
OPTIONS.update(
    iconfile="Icon.icns",
    plist=dict(CFBundleIdentifier='com.example.yourdomain.notmine')
)

setup(
    name="Your Game",
    app=APP,
    data_files=DATA_FILES,
    include_package_data=True,
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
    packages=pkgs,
    package_data={
        "": ["*.gal" , "*.gif" , "*.html" , "*.jar" , "*.js" , "*.mid" ,
             "*.png" , "*.py" , "*.pyc" , "*.sh" , "*.tmx" , "*.ttf" ,
             # "*.xcf"
        ]
    },
)

This is definitely somewhat less efficient than py2app's default of stuffing the code into a single zip file, but, as a counterpoint to that: it actually works.

Step 3: Build it

Hopefully, at this point you can just do python setup.py py2app and get a shiny new app bundle in dist/$NAME.app. We haven't had to go through the hell of quarantine just yet, so it should launch at this point. If it doesn't, sorry :-(.

You can often debug more obvious fail-to-launch issues by running the executable in the command line, by running ./dist/$NAME.app/Contents/MacOS/$NAME. Although this will run in a slightly different environment than double clicking (it will have all your shell's env vars, for example, so if your app needs an env var to work it might mysteriously work there) it will also print out any tracebacks to your terminal, where they'll be slightly easier to find than in Console.app.

Once your app at least runs locally, it's time to...

Step 4: Code sign it

All the tutorials that I've found on how to do this involve doing Xcode project goop where it's not clear what's happening underneath. But despite the fact that the introductory docs aren't quite there, the underlying model for codesigning stuff is totally common across GUI and command-line cases. However, actually getting your cert requires Xcode, an apple ID, and a credit card.

After paying your hundred dollars, go into Xcode, go to Accounts, hit "+", "Apple ID", then log in. Then, in your shiny new account, go to "Manage Certificates", hit the little "+", and (assuming, like me, you want to put something up on your own website, and not submit to the Mac App Store), and choose Developer ID Application. You probably think you want "mac app distribution" because you are wanting to distribute a mac app! But you don't.

Next, before you do anything else, make sure you have backups of your certificate and private key. You really don't want to lose the private key associated with that cert.

Now quit Xcode; you're done with the GUI.

You will need to know the identifier of your signing key though, which should be output from the command:

1
$ security find-identity -v -p codesigning | grep 'Developer ID' | sed -e 's/.*"\(.*\)"/\1/'

You probably want to put that in your build script, since you want to sign with the same identity every time. The command to do the signing is:

1
$ codesign -fs "${YOUR_DEVELOPER_ID_IDENTITY}" --deep "dist/${NAME}.app"

Step 5: Archive it

The right way to do this is probably to use dmgbuild or something like it, but what I promised here was quick and dirty, not beautiful and best practices.

You have to make an archive that preserves symbolic links. There are a few options for this:

Most importantly, if you use the zip command line tool, you must use the -y option. Without it, your downloadable app bundle will be somewhat mysteriously broken even though the one before you zipped it will be fine.

Step 6: Download it

Ideally, at this point, everything should be working. But to make sure that code-signing and archiving went correctly, you should have either a pristine virtual machine with no dev tools and no Python installed, or a non-programmer friend's machine that can serve the same purpose. They probably need a relatively recent macOS - I know that apps made using the above technique will definitely work on High Sierra and will definitely break on Yosemite; they probably start working at some OS version between those.

There's no tooling that I know of that can clearly tell you whether your mac app depends on some detail of your local machine. Even for your dependencies, there's no auditwheel for macOS. So it's always a good idea to check your final app build on a fresh computer before you announce it.

Coda

If you were expecting to get to the end and download my cool game, sorry to disappoint! It really is a half-broken prototype that is in no way ready for public consumption, and given my current load of personal and professional responsibilities, you definitely shouldn't expect anything from me in this area any time soon, or, you know, ever.

But, from years of experience, I know that it's nearly impossible to summon any motivation to work on small projects like this without the knowledge that the end result will be usable in some way, so I hope that this helps someone else set up their Python game-dev pipeline.

I'd really like to turn this into a 3-part series, with a part for Linux (perhaps using flatpak? is that a good thing?) and a part for Windows. However, given my aforementioned time constraints, I don't think I'm going to have the time or energy to do that research, so if you've got the appropriate knowledge, I'd love to host a guest post on this blog, or even just a link to yours.

If this post helped you, if you have questions or corrections, or if you'd like to write the Linux or Windows version of this post, let me know.

07 Jan 2018 8:48pm GMT