What's a good maximum line length for your coding standard?

This is, of course, a trick question. By posing it as a question, I have created the misleading impression that it is a question, but Black has selected the correct number for you; it's 88 which is obviously very lucky.

Thanks for reading my blog.

OK, OK. Clearly, there's more to it than that. This is an age-old debate on the level of "tabs versus spaces". So contentious, in fact, that even the famously opinionated Black does in fact let you change it.

Ancient History

One argument that certain silly people¹ like to make is "why are we wrapping at 80 characters like we are using 80 character teletypes, it's the 2020s! I have an ultrawide monitor!". The implication here is that the width of 80-character terminals is an antiquated relic, based entirely around the hardware limitations of a bygone era, and modern displays can put tons of stuff on one line, so why not use that capability?

This feels intuitively true, given the huge disparity between ancient times and now: on my own display, I can comfortably fit about 350 characters on a line. What a shame, to have so much room for so many characters in each line, and to waste it all on blank space!

But... is that true?

I stretched out my editor window all the way to measure that '350' number, but I did not continue editing at that window width. In order to have a more comfortable editing experience, I switched back into writeroom mode, a mode which emulates a considerably more writerly application, which limits each line length to 92 characters, regardless of frame width.

You've probably noticed this too. Almost all sites that display prose of any kind limit their width, even on very wide screens.

As silly as that tiny little ribbon of text running down the middle of your monitor might look with a full-screened stereotypical news site or blog, if you full-screen a site that doesn't set that width-limit, although it makes sense that you can now use all that space up, it will look extremely, almost unreadably bad.

Blogging software does not set a column width limit on your text because of some 80-character-wide accident of history in the form of a hardware terminal.

Similarly, if you really try to use that screen real estate to its fullest for coding, and start editing 200-300 character lines, you'll quickly notice it starts to feel just a bit weird and confusing. It gets surprisingly easy to lose your place. Rhetorically the "80 characters is just because of dinosaur technology! Use all those ultrawide pixels!" talking point is quite popular, but practically people usually just want a few more characters worth of breathing room, maxing out at 100 characters, far narrower than even the most svelte widescreen.

So maybe those 80 character terminals are holding us back a little bit, but... wait a second. Why were the terminals 80 characters wide in the first place?

Ancienter History

As this lovely Software Engineering Stack Exchange post summarizes, terminals were probably 80 characters because teletypes were 80 characters, and teletypes were probably 80 characters because punch cards were 80 characters, and punch cards were probably 80 characters because that's just about how many typewritten characters fit onto one line of a US-Letter piece of paper.

Even before typewriters, consider the average newspaper: why do we call a regularly-occurring featured article in a newspaper a "column"? Because broadsheet papers were too wide to have only a single column; they would always be broken into multiple! Far more aggressive than 80 characters, columns in newspapers typically have 30 characters per line.

The first newspaper printing machines were custom designed and could have used whatever width they wanted, so why standardize on something so narrow?³

Science!

There has been a surprising amount of scientific research around this issue, but in brief, there's a reason here rooted in human physiology: when you read a block of text, you are not consciously moving your eyes from word to word like you're dragging a mouse cursor, repositioning continuously. Human eyes reading text move in quick bursts of rotation called "saccades". In order to quickly and accurately move from one line of text to another, the start of the next line needs to be clearly visible in the reader's peripheral vision in order for them to accurately target it. This limits the angle of rotation that the reader can perform in a single saccade, and, thus, the length of a line that they can comfortably read without hunting around for the start of the next line every time they get to the end.

So, 80 (or 88) characters isn't too unreasonable for a limit. It's longer than 30 characters, that's for sure!

But, surely that's not all, or this wouldn't be so contentious in the first place?

Caveats

The screen is wide, though.

The ultrawide aficionados do have a point, even if it's not really the simple one about "old terminals" they originally thought. Our modern wide-screen displays are criminally underutilized, particularly for text. Even adding in the big chunky file, class, and method tree browser over on the left and the source code preview on the right, a brief survey of a Google Image search for "vs code" shows a lot of editors open with huge, blank areas on the right side of the window.

Big screens are super useful as they allow us to leverage our spatial memories to keep more relevant code around and simply glance around as we think, rather than navigate interactively. But it only works if you remember to do it.

Newspapers allowed us to read a ton of information in one sitting with minimum shuffling by packing in as much as 6 columns of text. You could read a column to the bottom of the page, back to the top, and down again, several times.

Similarly, books fill both of their opposed pages with text at the same time, doubling the amount of stuff you can read at once before needing to turn the page.

You may notice that reading text in a book, even in an ebook app, is more comfortable than reading a ton of text by scrolling around in a web browser. That's because our eyes are built for saccades, and repeatedly tracking the continuous smooth motion of the page as it scrolls to a stop, then re-targeting the new fixed location to start saccading around from, is literally more physically strenuous on your eye's muscles!

There's a reason that the codex was a big technological innovation over the scroll. This is a regression!

Today, the right thing to do here is to make use of horizontally split panes in your text editor or IDE, and just make a bit of conscious effort to set up the appropriate code on screen for the problem you're working on. However, this is a potential area for different IDEs to really differentiate themselves, and build multi-column continuous-code-reading layouts that allow for buffers to wrap and be navigable newspaper-style.

Similar, modern CSS has shockingly good support for multi-column layouts, and it's a shame that true multi-column, page-turning layouts are so rare. If I ever figure out a way to deploy this here that isn't horribly clunky and fighting modern platform conventions like "scrolling horizontally is substantially more annoying and inconsistent than scrolling vertically" maybe I will experiment with such a layout on this blog one day. Until then… just make the browser window narrower so other useful stuff can be in the other parts of the screen, I guess.

Code Isn't Prose

But, I digress. While I think that columnar layouts for reading prose are an interesting thing more people should experiment with, code isn't prose.

The metric used for ideal line width, which you may have noticed if you clicked through some of those Wikipedia links earlier, is not "character cells in your editor window", it is characters per line, or "CPL".

With an optimal CPL somewhere between 45 and 95, a code-line-width of somewhere around 90 might actually be the best idea, because whitespace uses up your line-width budget. In a typical object-oriented Python program², most of your code ends up indented by at least 8 spaces: 4 for the class scope, 4 for the method scope. Most likely a lot of it is 12, because any interesting code will have at least one conditional or loop. So, by the time you're done wasting all that horizontal space, a max line length of 90 actually looks more like a maximum of 78... right about that sweet spot from the US-Letter page in the typewriter that we started with.

What about soft-wrap?

In principle, source code is structured information, whose presentation could be fully decoupled from its serialized representation. Everyone could configure their preferred line width appropriate to their custom preferences and the specific physiological characteristics of their eyes, and the code could be formatted according to the language it was expressed in, and "hard wrapping" could be a silly antiquated thing.

The problem with this argument is the same as the argument against "but tabs are semantic indentation", to wit: nope, no it isn't. What "in principle" means in the previous paragraph is actually "in a fantasy world which we do not inhabit". I'd love it if editors treated code this way and we had a rich history and tradition of structured manipulations rather than typing in strings of symbols to construct source code textually. But that is not the world we live in. Hard wrapping is unfortunately necessary to integrate with diff tools.

So what's the optimal line width?

The exact, specific number here is still ultimately a matter of personal preference.

Hopefully, understanding the long history, science, and underlying physical constraints can lead you to select a contextually appropriate value for your own purposes that will balance ease of reading, integration with the relevant tools in your ecosystem, diff size, presentation in the editors and IDEs that your contributors tend to use, reasonable display in web contexts, on presentation slides, and so on.

But - and this is important - counterpoint:

No it isn't, you don't need to select an optimal width, because it's already been selected for you. It is 88.

Acknowledgments

Thank you for reading, and especially thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

08 Aug 2025 5:37am GMT

The Problem

Like many other self-styled thinky programmer guys, I like to imagine myself as a sort of Holmesian genius, making trenchant observations, collecting them, and then synergizing them into brilliant deductions with the keen application of my powerful mind.

However, several years ago, I had an epiphany in my self-concept. I finally understood that, to the extent that I am usefully clever, it is less in a Holmesian idiom, and more, shall we say, Monkesque.

For those unfamiliar with either of the respective franchises:

• Holmes is a towering intellect honed by years of training, who catalogues intentional, systematic observations and deduces logical, factual conclusions from those observations.
• Monk, on the other hand, while also a reasonably intelligent guy, is highly neurotic, wracked by unresolved trauma and profound grief. As both a consulting job and a coping mechanism, he makes a habit of erratically wandering into crime scenes, and, driven by a carefully managed jenga tower of mental illnesses, leverages his dual inabilities to solve crimes. First, he is unable to filter out apparently inconsequential details, building up a mental rat's nest of trivia about the problem; second, he is unable to let go of any minor incongruity, obsessively ruminating on the collection of facts until they all make sense in a consistent timeline.

Perhaps surprisingly, this tendency serves both this fictional wretch of a detective, and myself, reasonably well. I find annoying incongruities in abstractions and I fidget and fiddle with them until I end up building something that a lot of people like, or perhaps something that a smaller number of people get really excited about. At worst, at least I eventually understand what's going on. This is a self-soothing activity but it turns out that, managed properly, it can very effectively soothe others as well.

All that brings us to today's topic, which is an incongruity I cannot smooth out or fit into a logical framework to make sense. I am, somewhat reluctantly, a genAI skeptic. However, I am, even more reluctantly, exposed to genAI Discourse every damn minute of every damn day. It is relentless, inescapable, and exhausting.

This preamble about personality should hopefully help you, dear reader, to understand how I usually address problematical ideas by thinking and thinking and fidgeting with them until I manage to write some words - or perhaps a new open source package - that logically orders the ideas around it in a way which allows my brain to calm down and let it go, and how that process is important to me.

In this particular instance, however, genAI has defeated me. I cannot make it make sense, but I need to stop thinking about it anyway. It is too much and I need to give up.

My goal with this post is not to convince anyone of anything in particular - and we'll get to why that is a bit later - but rather:

1. to set out my current understanding in one place, including all the various negative feelings which are still bothering me, so I can stop repeating it elsewhere,
2. to explain why I cannot build a case that I think should be particularly convincing to anyone else, particularly to someone who actively disagrees with me,
3. in so doing, to illustrate why I think the discourse is so fractious and unresolvable, and finally
4. to give myself, and hopefully by proxy to give others in the same situation, permission to just peace out of this nightmare quagmire corner of the noosphere.

But first, just because I can't prove that my interlocutors are Wrong On The Internet, doesn't mean I won't explain why I feel like they are wrong.

The Anti-Antis

Most recently, at time of writing, there have been a spate of "the genAI discourse is bad" articles, almost exclusively written from the perspective of, not boosters exactly, but pragmatically minded (albeit concerned) genAI users, wishing for the skeptics to be more pointed and accurate in our critiques. This is anti-anti-genAI content.

I am not going to link to any of these, because, as part of their self-fulfilling prophecy about the "genAI discourse", they're also all bad.

Mostly, however, they had very little worthwhile to respond to because they were straw-manning their erstwhile interlocutors. They are all getting annoyed at "bad genAI criticism" while failing to engage with - and often failing to even mention - most of the actual substance of any serious genAI criticism. At least, any of the criticism that I've personally read.

I understand wanting to avoid a callout or Gish-gallop culture and just express your own ideas. So, I understand that they didn't link directly to particular sources or go point-by-point on anyone else's writing. Obviously I get it, since that's exactly what this post is doing too.

But if you're going to talk about how bad the genAI conversation is, without even mentioning huge categories of problem like "climate impact" or "disinformation"¹ even once, I honestly don't know what conversation you're even talking about. This is peak "make up a guy to get mad at" behavior, which is especially confusing in this circumstance, because there's an absolutely huge crowd of actual people that you could already be mad at.

The people writing these pieces have historically seemed very thoughtful to me. Some of them I know personally. It is worrying to me that their critical thinking skills appear to have substantially degraded specifically after spending a bunch of time intensely using this technology which I believe has a scary risk of degrading one's critical thinking skills. Correlation is not causation or whatever, and sure, from a rhetorical perspective this is "post hoc ergo propter hoc" and maybe a little "ad hominem" for good measure, but correlation can still be concerning.

Yet, I cannot effectively respond to these folks, because they are making a practical argument that I cannot, despite my best efforts, find compelling evidence to refute categorically. My experiences of genAI are all extremely bad, but that is barely even anecdata. Their experiences are neutral-to-positive. Little scientific data exists. How to resolve this?²

The Aesthetics

As I begin to state my own position, let me lead with this: my factual analysis of genAI is hopelessly negatively biased. I find the vast majority of the aesthetic properties of genAI to be intensely unpleasant.

I have been trying very hard to correct for this bias, to try to pay attention to the facts and to have a clear-eyed view of these systems' capabilities. But the feelings are visceral, and the effort to compensate is tiring. It is, in fact, the desire to stop making this particular kind of effort that has me writing up this piece and trying to take an intentional break from the subject, despite its intense relevance.

When I say its "aesthetic qualities" are unpleasant, I don't just mean the aesthetic elements of output of genAIs themselves. The aesthetic quality of genAI writing, visual design, animation and so on, while mostly atrocious, is also highly variable. There are cherry-picked examples which look… fine. Maybe even good. For years now, there have been, famously, literally award-winning aesthetic outputs of genAI³.

While I am ideologically predisposed to see any "good" genAI art as accruing the benefits of either a survivorship bias from thousands of terrible outputs or simple plagiarism rather than its own inherent quality, I cannot deny that in many cases it is "good".

However, I am not just talking about the product, but the process; the aesthetic experience of interfacing with the genAI system itself, rather than the aesthetic experience of the outputs of that system.

I am not a visual artist and I am not really a writer⁴, particularly not a writer of fiction or anything else whose experience is primarily aesthetic. So I will speak directly to the experience of software development.

I have seen very few successful examples of using genAI to produce whole, working systems. There are no shortage of highly public miserable failures, particularly from the vendors of these systems themselves, where the outputs are confused, self-contradictory, full of subtle errors and generally unusable. While few studies exist, it sure looks like this is an automated way of producing a Net Negative Productivity Programmer, throwing out chaff to slow down the rest of the team.⁵

Juxtapose this with my aforementioned psychological motivations, to wit, I want to have everything in the computer be orderly and make sense, I'm sure most of you would have no trouble imagining that sitting through this sort of practice would make me extremely unhappy.

Despite this plethora of negative experiences, executives are aggressively mandating the use of AI⁶. It looks like without such mandates, most people will not bother to use such tools, so the executives will need muscular policies to enforce its use.⁷

Being forced to sit and argue with a robot while it struggles and fails to produce a working output, while you have to rewrite the code at the end anyway, is incredibly demoralizing. This is the kind of activity that activates every single major cause of burnout at once.

But, at least in that scenario, the thing ultimately doesn't work, so there's a hope that after a very stressful six month pilot program, you can go to management with a pile of meticulously collected evidence, and shut the whole thing down.

I am inclined to believe that, in fact, it doesn't work well enough to be used this way, and that we are going to see a big crash. But that is not the most aesthetically distressing thing. The most distressing thing is that maybe it does work; if not well enough to actually do the work, at least ambiguously enough to fool the executives long-term.

This project, in particular, stood out to me as an example. Its author, a self-professed "AI skeptic" who "thought LLMs were glorified Markov chain generators that didn't actually understand code and couldn't produce anything novel", did a green-field project to test this hypothesis.

Now, this particular project is not totally inconsistent with a world in which LLMs cannot produce anything novel. One could imagine that, out in the world of open source, perhaps there is enough "OAuth provider written in TypeScript" blended up into the slurry of "borrowed⁸" training data that the minor constraint of "make it work on Cloudflare Workers" is a small tweak⁹. It is not fully dispositive of the question of the viability of "genAI coding".

But it is a data point related to that question, and thus it did make me contend with what might happen if it were actually a fully demonstrative example. I reviewed the commit history, as the author suggested. For the sake of argument, I tried to ask myself if I would like working this way. Just for clarity on this question, I wanted to suspend judgement about everything else; assuming:

• the model could be created with ethically, legally, voluntarily sourced training data
• its usage involved consent from labor rather than authoritarian mandates
• sensible levels of energy expenditure, with minimal CO2 impact
• it is substantially more efficient to work this way than to just write the code yourself

and so on, and so on… would I like to use this magic robot that could mostly just emit working code for me? Would I use it if it were free, in all senses of the word?

No. I absolutely would not.

I found the experience of reading this commit history and imagining myself using such a tool - without exaggeration - nauseating.

Unlike many programmers, I love code review. I find that it is one of the best parts of the process of programming. I can help people learn, and develop their skills, and learn from them, and appreciate the decisions they made, develop an impression of a fellow programmer's style. It's a great way to build a mutual theory of mind.

Of course, it can still be really annoying; people make mistakes, often can't see things I find obvious, and in particular when you're reviewing a lot of code from a lot of different people, you often end up having to repeat explanations of the same mistakes. So I can see why many programmers, particularly those more introverted than I am, hate it.

But, ultimately, when I review their code and work hard to provide clear and actionable feedback, people learn and grow and it's worth that investment in inconvenience.

The process of coding with an "agentic" LLM appears to be the process of carefully distilling all the worst parts of code review, and removing and discarding all of its benefits.

The lazy, dumb, lying robot asshole keeps making the same mistakes over and over again, never improving, never genuinely reacting, always obsequiously pretending to take your feedback on board.

Even when it "does" actually "understand" and manages to load your instructions into its context window, 200K tokens later it will slide cleanly out of its memory and you will have to say it again.

All the while, it is attempting to trick you. It gets most things right, but it consistently makes mistakes in the places that you are least likely to notice. In places where a person wouldn't make a mistake. Your brain keeps trying to develop a theory of mind to predict its behavior but there's no mind there, so it always behaves infuriatingly randomly.

I don't think I am the only one who feels this way.

The Affordances

Whatever our environments afford, we tend to do more of. Whatever they resist, we tend to do less of. So in a world where we were all writing all of our code and emails and blog posts and texts to each other with LLMs, what do they afford that existing tools do not?

As a weirdo who enjoys code review, I also enjoy process engineering. The central question of almost all process engineering is to continuously ask: how shall we shape our tools, to better shape ourselves?

LLMs are an affordance for producing more text, faster. How is that going to shape us?

Again arguing in the alternative here, assuming the text is free from errors and hallucinations and whatever, it's all correct and fit for purpose, that means it reduces the pain of circumstances where you have to repeat yourself. Less pain! Sounds great; I don't like pain.

Every codebase has places where you need boilerplate. Every organization has defects in its information architecture that require repetition of certain information rather than a link back to the authoritative source of truth. Often, these problems persist for a very long time, because it is difficult to overcome the institutional inertia required to make real progress rather than going along with the status quo. But this is often where the highest-value projects can be found. Where there's muck, there's brass.

The process-engineering function of an LLM, therefore, is to prevent fundamental problems from ever getting fixed, to reward the rapid-fire overwhelm of infrastructure teams with an immediate, catastrophic cascade of legacy code that is now much harder to delete than it is to write.

There is a scene in Game of Thrones where Khal Drogo kills himself. He does so by replacing a stinging, burning, therapeutic antiseptic wound dressing with some cool, soothing mud. The mud felt nice, addressed the immediate pain, removed the discomfort of the antiseptic, and immediately gave him a lethal infection.

The pleasing feeling of immediate progress when one prompts an LLM to solve some problem feels like cool mud on my brain.

The Economics

We are in the middle of a mania around this technology. As I have written about before, I believe the mania will end. There will then be a crash, and a "winter". But, as I may not have stressed sufficiently, this crash will be the biggest of its kind - so big, that it is arguably not of a kind at all. The level of investment in these technologies is bananas and the possibility that the investors will recoup their investment seems close to zero. Meanwhile, that cost keeps going up, and up, and up.

Others have reported on this in detail¹⁰, and I will not reiterate that all here, but in addition to being a looming and scary industry-wide (if we are lucky; more likely it's probably "world-wide") economic threat, it is also going to drive some panicked behavior from management.

Panicky behavior from management stressed that their idea is not panning out is, famously, the cause of much human misery. I expect that even in the "good" scenario, where some profit is ultimately achieved, will still involve mass layoffs rocking the industry, panicked re-hiring, destruction of large amounts of wealth.

It feels bad to think about this.

The Energy Usage

For a long time I believed that the energy impact was overstated. I am even on record, about a year ago, saying I didn't think the energy usage was a big deal. I think I was wrong about that.

It initially seemed like it was letting regular old data centers off the hook. But recently I have learned that, while the numbers are incomplete because the vendors aren't sharing information, they're also extremely bad.¹¹

I think there's probably a version of this technology that isn't a climate emergency nightmare, but that's not the version that the general public has access to today.

The Educational Impact

LLMs are making academic cheating incredibly rampant.¹²

Not only is it so common as to be nearly universal, it's also extremely harmful to learning.¹³

For learning, genAI is a forklift at the gym.

To some extent, LLMs are simply revealing a structural rot within education and academia that has been building for decades if not centuries. But it was within those inefficiencies and the inconveniences of the academic experience that real learning was, against all odds, still happening in schools.

LLMs produce a frictionless, streamlined process where students can effortlessly glide through the entire credential, learning nothing. Once again, they dull the pain without regard to its cause.

This is not good.

The Invasion of Privacy

This is obviously only a problem with the big cloud models, but then, the big cloud models are the only ones that people actually use. If you are having conversations about anything private with ChatGPT, you are sending all of that private information directly to Sam Altman, to do with as he wishes.

Even if you don't think he is a particularly bad guy, maybe he won't even create the privacy nightmare on purpose. Maybe he will be forced to do so as a result of some bizarre kafkaesque accident.¹⁴

Imagine the scenario, for example, where a woman is tracking her cycle and uploading the logs to ChatGPT so she can chat with it about a health concern. Except, surprise, you don't have to imagine, you can just search for it, as I have personally, organically, seen three separate women on YouTube, at least one of whom lives in Texas, not only do this on camera but recommend doing this to their audiences.

Citation links withheld on this particular claim for hopefully obvious reasons.

I assure you that I am neither particularly interested in menstrual products nor genAI content, and if I am seeing this more than once, it is probably a distressingly large trend.

The Stealing

The training data for LLMs is stolen. I don't mean like "pirated" in the sense where someone illicitly shares a copy they obtained legitimately; I mean their scrapers are ignoring both norms¹⁵ and laws¹⁶ to obtain copies under false pretenses, destroying other people's infrastructure¹⁷ in the process.

The Fatigue

I have provided references to numerous articles outlining rhetorical and sometimes data-driven cases for the existence of certain properties and consequences of genAI tools. But I can't prove any of these properties, either at a point in time or as a durable ongoing problem.

The LLMs themselves are simply too large to model with the usual kind of heuristics one would use to think about software. I'd sooner be able to predict the physics of dice in a casino than a 2 trillion parameter neural network. They resist scientific understanding, not just because of their size and complexity, but because unlike a natural phenomenon (which could of course be considerably larger and more complex) they resist experimentation.

The first form of genAI resistance to experiment is that every discussion is a motte-and-bailey. If I use a free model and get a bad result I'm told it's because I should have used the paid model. If I get a bad result with ChatGPT I should have used Claude. If I get a bad result with a chatbot I need to start using an agentic tool. If an agentic tool deletes my hard drive by putting os.system("rm -rf ~/") into sitecustomize.py then I guess I should have built my own MCP integration with a completely novel heretofore never even considered security sandbox or something?

What configuration, exactly, would let me make a categorical claim about these things? What specific methodological approach should I stick to, to get reliably adequate prompts?

For the record though, if the idea of the free models is that they are going to be provocative demonstrations of the impressive capabilities of the commercial models, and the results are consistently dogshit, I am finding it increasingly hard to care how much better the paid ones are supposed to be, especially since the "better"-ness cannot really be quantified in any meaningful way.

The motte-and-bailey doesn't stop there though. It's a war on all fronts. Concerned about energy usage? That's OK, you can use a local model. Concerned about infringement? That's okay, somewhere, somebody, maybe, has figured out how to train models consensually¹⁸. Worried about the politics of enriching the richest monsters in the world? Don't worry, you can always download an "open source" model from Hugging Face. It doesn't matter that many of these properties are mutually exclusive and attempting to fix one breaks two others; there's always an answer, the field is so abuzz with so many people trying to pull in so many directions at once that it is legitimately difficult to understand what's going on.

Even here though, I can see that characterizing everything this way is unfair to a hypothetical sort of person. If there is someone working at one of these thousands of AI companies that have been springing up like toadstools after a rain, and they really are solving one of these extremely difficult problems, how can I handwave that away? We need people working on problems, that's like, the whole point of having an economy. And I really don't like shitting on other people's earnest efforts, so I try not to dismiss whole fields. Given how AI has gotten into everything, in a way that e.g. cryptocurrency never did, painting with that broad a brush inevitably ends up tarring a bunch of stuff that isn't even really AI at all.

The second form of genAI resistance to experiment is the inherent obfuscation of productization. The models themselves are already complicated enough, but the products that are built around the models are evolving extremely rapidly. ChatGPT is not just a "model", and with the rapid¹⁹ deployment of Model Context Protocol tools, the edges of all these things will blur even further. Every LLM is now just an enormous unbounded soup of arbitrary software doing arbitrary whatever. How could I possibly get my arms around that to understand it?

The Challenge

I have woefully little experience with these tools.

I've tried them out a little bit, and almost every single time the result has been a disaster that has not made me curious to push further. Yet, I keep hearing from all over the industry that I should.

To some extent, I feel like the motte-and-bailey characterization above is fair; if the technology itself can really do real software development, it ought to be able to do it in multiple modalities, and there's nothing anyone can articulate to me about GPT-4o which puts it in a fundamentally different class than GPT-3.5.

But, also, I consistently hear that the subjective experience of using the premium versions of the tools is actually good, and the free ones are actually bad.

I keep struggling to find ways to try them "the right way", the way that people I know and otherwise respect claim to be using them, but I haven't managed to do so in any meaningful way yet.

I do not want to be using the cloud versions of these models with their potentially hideous energy demands; I'd like to use a local model. But there is obviously not a nicely composed way to use local models like this.

Since there are apparently zero models with ethically-sourced training data, and litigation is ongoing²⁰ to determine the legal relationships of training data and outputs, even if I can be comfortable with some level of plagiarism on a project, I don't feel that I can introduce the existential legal risk into other people's infrastructure, so I would need to make a new project.

Others have differing opinions of course, including some within my dependency chain, which does worry me, but I still don't feel like I can freely contribute further to the problem; it's going to be bad enough to unwind any impact upstream. Even just for my own sake, I don't want to make it worse.

This especially presents a problem because I have way too much stuff going on already. A new project is not practical.

Finally, even if I did manage to satisfy all of my quirky²¹ constraints, would this experiment really be worth anything? The models and tools that people are raving about are the big, expensive, harmful ones. If I proved to myself yet again that a small model with bad tools was unpleasant to use, I wouldn't really be addressing my opponents' views.

I'm stuck.

The Surrender

I am writing this piece to make my peace with giving up on this topic, at least for a while. While I do idly hope that some folks might find bits of it convincing, and perhaps find ways to be more mindful with their own usage of genAI tools, and consider the harm they may be causing, that's not actually the goal. And that is not the goal because it is just so much goddamn work to prove.

Here, I must return to my philosophical hobbyhorse of sprachspiel. In this case, specifically to use it as an analytical tool, not just to understand what I am trying to say, but what the purpose for my speech is.

The concept of sprachspiel is most frequently deployed to describe the goal of the language game being played, but in game theory, that's only half the story. Speech - particularly rigorously justified speech - has a cost, as well as a benefit. I can make shit up pretty easily, but if I want to do anything remotely like scientific or academic rigor, that cost can be astronomical. In the case of developing an abstract understanding of LLMs, the cost is just too high.

So what is my goal, then? To be king Canute, standing astride the shore of "tech", whatever that is, commanding the LLM tide not to rise? This is a multi-trillion dollar juggernaut.

Even the rump, loser, also-ran fragment of it has the power to literally suffocate us in our homes²² if they so choose, completely insulated from any consequence. If the power curve starts there, imagine what the winners in this industry are going to be capable of, irrespective of the technology they're building - just with the resources they have to hand. Am I going to write a blog post that can rival their propaganda apparatus? Doubtful.

Instead, I will just have to concede that maybe I'm wrong. I don't have the skill, or the knowledge, or the energy, to demonstrate with any level of rigor that LLMs are generally, in fact, hot garbage. Intellectually, I will have to acknowledge that maybe the boosters are right. Maybe it'll be OK.

Maybe the carbon emissions aren't so bad. Maybe everybody is keeping them secret in ways that they don't for other types of datacenter for perfectly legitimate reasons. Maybe the tools really can write novel and correct code, and with a little more tweaking, it won't be so difficult to get them to do it. Maybe by the time they become a mandatory condition of access to developer tools, they won't be miserable.

Sure, I even sincerely agree, intellectual property really has been a pretty bad idea from the beginning. Maybe it's OK that we've made an exception to those rules. The rules were stupid anyway, so what does it matter if we let a few billionaires break them? Really, everybody should be able to break them (although of course, regular people can't, because we can't afford the lawyers to fight off the MPAA and RIAA, but that's a problem with the legal system, not tech).

I come not to praise "AI skepticism", but to bury it.

Maybe it really is all going to be fine. Perhaps I am simply catastrophizing; I have been known to do that from time to time. I can even sort of believe it, in my head. Still, even after writing all this out, I can't quite manage to believe it in the pit of my stomach.

Unfortunately, that feeling is not something that you, or I, can argue with.

Acknowledgments

Thank you to my patrons. Normally, I would say, "who are supporting my writing on this blog", but in the case of this piece, I feel more like I should apologize to them for this than to thank them; these thoughts have been preventing me from thinking more productive, useful things that I actually have relevant skill and expertise in; this felt more like a creative blockage that I just needed to expel than a deliberately written article. If you like what you've read here and you'd like to read more of it, well, too bad; I am sincerely determined to stop writing about this topic. But, if you'd like to read more stuff like other things I have written, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

05 Jun 2025 5:22am GMT

The History

Before dataclasses were added to Python in version 3.7 - in June of 2018 - the __init__ special method had an important use. If you had a class representing a data structure - for example a 2DCoordinate, with x and y attributes - you would want to be able to construct it as 2DCoordinate(x=1, y=2), which would require you to add an __init__ method with x and y parameters.

The other options available at the time all had pretty bad problems:

1. You could remove 2DCoordinate from your public API and instead expose a make_2d_coordinate function and make it non-importable, but then how would you document your return or parameter types?
2. You could document the x and y attributes and make the user assign each one themselves, but then 2DCoordinate() would return an invalid object.
3. You could default your coordinates to 0 with class attributes, and while that would fix the problem with option 2, this would now require all 2DCoordinate objects to be not just mutable, but mutated at every call site.
4. You could fix the problems with option 1 by adding a new abstract class that you could expose in your public API, but this would explode the complexity of every new public class, no matter how simple. To make matters worse, typing.Protocol didn't even arrive until Python 3.8, so, in the pre-3.7 world this would condemn you to using concrete inheritance and declaring multiple classes even for the most basic data structure imaginable.

Also, an __init__ method that does nothing but assign a few attributes doesn't have any significant problems, so it is an obvious choice in this case. Given all the problems that I just described with the alternatives, it makes sense that it became the obvious default choice, in most cases.

However, by accepting "define a custom __init__" as the default way to allow users to create your objects, we make a habit of beginning every class with a pile of arbitrary code that gets executed every time it is instantiated.

Wherever there is arbitrary code, there are arbitrary problems.

The Problems

Let's consider a data structure more complex than one that simply holds a couple of attributes. We will create one that represents a reference to some I/O in the external world: a FileReader.

Of course Python has its own open-file object abstraction, but I will be ignoring that for the purposes of the example.

Let's assume a world where we have the following functions, in an imaginary fileio module:

• open(path: str) -> int
• read(fileno: int, length: int)
• close(fileno: int)

Our hypothetical fileio.open returns an integer representing a file descriptor¹, fileio.read allows us to read length bytes from an open file descriptor, and fileio.close closes that file descriptor, invalidating it for future use.

With the habit that we have built from writing thousands of __init__ methods, we might want to write our FileReader class like this:

1
2
3
4
5
6
7

class FileReader:
    def __init__(self, path: str) -> None:
        self._fd = fileio.open(path)
    def read(self, length: int) -> bytes:
        return fileio.read(self._fd, length)
    def close(self) -> None:
        fileio.close(self._fd)

For our initial use-case, this is fine. Client code creates a FileReader by doing something like FileReader("./config.json"), which always creates a FileReader that maintains its file descriptor int internally as private state. This is as it should be; we don't want user code to see or mess with _fd, as that might violate FileReader's invariants. All the necessary work to construct a valid FileReader - i.e. the call to open - is always taken care of for you by FileReader.__init__.

However, additional requirements will creep in, and as they do, FileReader.__init__ becomes increasingly awkward.

Initially we only care about fileio.open, but later, we may have to deal with a library that has its own reasons for managing the call to fileio.open by itself, and wants to give us an int that we use as our _fd, we now have to resort to weird workarounds like:

1
2
3
4

def reader_from_fd(fd: int) -> FileReader:
    fr = object.__new__(FileReader)
    fr._fd = fd
    return fr

Now, all those nice properties that we got from trying to force object construction to give us a valid object are gone. reader_from_fd's type signature, which takes a plain int, has no way of even suggesting to client code how to ensure that it has passed in the right kind of int.

Testing is much more of a hassle, because we have to patch in our own copy of fileio.open any time we want an instance of a FileReader in a test without doing any real-life file I/O, even if we could (for example) share a single file descriptor among many FileReader s for testing purposes.

All of this also assumes a fileio.open that is synchronous. Although for literal file I/O this is more of a hypothetical concern, there are many types of networked resource which are really only available via an asynchronous (and thus: potentially slow, potentially error-prone) API. If you've ever found yourself wanting to type async def __init__(self): ... then you have seen this limitation in practice.

Comprehensively describing all the possible problems with this approach would end up being a book-length treatise on a philosophy of object oriented design, so I will sum up by saying that the cause of all these problems is the same: we are inextricably linking the act of creating a data structure with whatever side-effects are most often associated with that data structure. If they are "often" associated with it, then by definition they are not "always" associated with it, and all the cases where they aren't associated become unweildy and potentially broken.

Defining an __init__ is an anti-pattern, and we need a replacement for it.

The Solutions

I believe this tripartite assemblage of design techniques will address the problems raised above:

• using dataclass to define attributes,
• replacing behavior that previously would have previously been in __init__ with a new classmethod that does the same thing, and
• using precise types to describe what a valid instance looks like.

Using dataclass attributes to create an __init__ for you

To begin, let's refactor FileReader into a dataclass. This does get us an __init__ method, but it won't be one an arbitrary one we define ourselves; it will get the useful constraint enforced on it that it will just assign attributes.

1
2
3
4
5
6
7

@dataclass
class FileReader:
    _fd: int
    def read(self, length: int) -> bytes:
        return fileio.read(self._fd, length)
    def close(self) -> None:
        fileio.close(self._fd)

Except... oops. In fixing the problems that we created with our custom __init__ that calls fileio.open, we have re-introduced several problems that it solved:

1. We have removed all the convenience of FileReader("path"). Now the user needs to import the low-level fileio.open again, making the most common type of construction both more verbose and less discoverable; if we want users to know how to build a FileReader in a practical scenario, we will have to add something in our documentation to point at a separate module entirely.
2. There's no enforcement of the validity of _fd as a file descriptor; it's just some integer, which the user could easily pass an incorrect instance of, with no error.

In isolation, dataclass by itself can't solve all our problems, so let's add in the second technique.

Using classmethod factories to create objects

We don't want to require any additional imports, or require users to go looking at any other modules - or indeed anything other than FileReader itself - to figure out how to create a FileReader for its intended usage.

Luckily we have a tool that can easily address all of these concerns at once: @classmethod. Let's define a FileReader.open class method:

1
2
3
4
5
6
7

from typing import Self
@dataclass
class FileReader:
    _fd: int
    @classmethod
    def open(cls, path: str) -> Self:
        return cls(fileio.open(path))

Now, your callers can replace FileReader("path") with FileReader.open("path"), and get all the same benefits.

Additionally, if we needed to await fileio.open(...), and thus we needed its signature to be @classmethod async def open, we are freed from the constraint of __init__ as a special method. There is nothing that would prevent a @classmethod from being async, or indeed, from having any other modification to its return value, such as returning a tuple of related values rather than just the object being constructed.

Using NewType to address object validity

Next, let's address the slightly trickier issue of enforcing object validity.

Our type signature calls this thing an int, and indeed, that is unfortunately what the lower-level fileio.open gives us, and that's beyond our control. But for our own purposes, we can be more precise in our definitions, using NewType:

1
2

from typing import NewType
FileDescriptor = NewType("FileDescriptor", int)

There are a few different ways to address the underlying library, but for the sake of brevity and to illustrate that this can be done with zero run-time overhead, let's just insist to Mypy that we have versions of fileio.open, fileio.read, and fileio.write which actually already take FileDescriptor integers rather than regular ones.

1
2
3
4

from typing import Callable
_open: Callable[[str], FileDescriptor] = fileio.open  # type:ignore[assignment]
_read: Callable[[FileDescriptor, int], bytes] = fileio.read
_close: Callable[[FileDescriptor], None] = fileio.close

We do of course have to slightly adjust FileReader, too, but the changes are very small. Putting it all together, we get:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

from typing import Self
@dataclass
class FileReader:
    _fd: FileDescriptor
    @classmethod
    def open(cls, path: str) -> Self:
        return cls(_open(path))
    def read(self, length: int) -> bytes:
        return _read(self._fd, length)
    def close(self) -> None:
        _close(self._fd)

Note that the main technique here is not necessarily using NewType specifically, but rather aligning an instance's property of "has all attributes set" as closely as possible with an instance's property of "fully valid instance of its class"; NewType is just a handy tool to enforce any necessary constraints on the places where you need to use a primitive type like int, str or bytes.

In Summary - The New Best Practice

From now on, when you're defining a new Python class:

• Make it a dataclass².
• Use its default __init__ method³.
• Add @classmethods to provide your users convenient and discoverable ways to build your objects.
• Require that all dependencies be satisfied by attributes, so you always start with a valid object.
• Use typing.NewType to enforce any constraints on primitive data types (like int and str) which might have magical external attributes, like needing to come from a particular library, needing to be random, and so on.

If you define all your classes this way, you will get all the benefits of a custom __init__ method:

• All consumers of your data structures will receive valid objects, because an object with all its attributes populated correctly is inherently valid.
• Users of your library will be presented with convenient ways to create your objects that do as much work as is necessary to make them easy to use, and they can discover these just by looking at the methods on your class itself.

Along with some nice new benefits:

• You will be future-proofed against new requirements for different ways that users may need to construct your object.
• If there are already multiple ways to instantiate your class, you can now give each of them a meaningful name; no need to have monstrosities like def __init__(self, maybe_a_filename: int | str | None = None):
• Your test suite can always construct an object by satisfying all its dependencies; no need to monkey-patch anything when you can always call the type and never do any I/O or generate any side effects.

Before dataclasses, it was always a bit weird that such a basic feature of the Python language - giving data to a data structure to make it valid - required overriding a method with 4 underscores in its name. __init__ stuck out like a sore thumb. Other such methods like __add__ or even __repr__ were inherently customizing esoteric attributes of classes.

For many years now, that historical language wart has been resolved. @dataclass, @classmethod, and NewType give you everything you need to build classes which are convenient, idiomatic, flexible, testable, and robust.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor! I am also available for consulting work if you think your organization could benefit from expertise on topics like "but what is a 'class', really?".

17 Apr 2025 10:35pm GMT

A Database File

When I was 10 years old, and going through a fairly difficult time, I was lucky enough to come into the possession of a piece of software called Claris FileMaker Pro™.

FileMaker allowed its users to construct arbitrary databases, and to associate their tables with a customized visual presentation. FileMaker also had a rudimentary scripting language, which would allow users to imbue these databases with behavior.

As a mentally ill pre-teen, lacking a sense of control over anything or anyone in my own life, including myself, I began building a personalized database to catalogue the various objects and people in my immediate vicinity. If one were inclined to be generous, one might assess this behavior and say I was systematically taxonomizing the objects in my life and recording schematized information about them.

As I saw it at the time, if I collected the information, I could always use it later, to answer questions that I might have. If I didn't collect it, then what if I needed it? Surely I would regret it! Thus I developed a categorical imperative to spend as much of my time as possible collecting and entering data about everything that I could reasonably arrange into a common schema.

Having thus summoned this specter of regret for all lost data-entry opportunities, it was hard to dismiss. We might label it "Claris's Basilisk", for obvious reasons.

Therefore, a less-generous (or more clinically-minded) observer might have replaced the word "systematically" with "obsessively" in the assessment above.

I also began writing what scripts were within my marginal programming abilities at the time, just because I could: things like computing the sum of every street number of every person in my address book. Why was this useful? Wrong question: the right question is "was it possible" to which my answer was "yes".

If I was obliged to collect all the information which I could observe - in case it later became interesting - I was similarly obliged to write and run every program I could. It might, after all, emit some other interesting information.

I was an avid reader of science fiction as well.

I had this vague sense that computers could kind of think. This resulted in a chain of reasoning that went something like this:

1. human brains are kinda like computers,
2. the software running in the human brain is very complex,
3. I could only write simple computer programs, but,
4. when you really think about it, a "complex" program is just a collection of simpler programs

Therefore: if I just kept collecting data, collecting smaller programs that could solve specific problems, and connecting them all together in one big file, eventually the database as a whole would become self-aware and could solve whatever problem I wanted. I just needed to be patient; to "keep grinding" as the kids would put it today.

I still feel like this is an understandable way to think - if you are a highly depressed and anxious 10-year-old in 1990.

Anyway.

35 Years Later

OpenAI is a company that produces transformer architecture machine learning generative AI models; their current generation was trained on about 10 trillion words, obtained in a variety of different ways from a large variety of different, unrelated sources.

A few days ago, on March 26, 2025 at 8:41 AM Pacific Time, Sam Altman took to "X™, The Everything App™," and described the trajectory of his career of the last decade at OpenAI as, and I quote, a "grind for a decade trying to help make super-intelligence to cure cancer or whatever" (emphasis mine).

I really, really don't want to become a full-time AI skeptic, and I am not an expert here, but I feel like I can identify a logically flawed premise when I see one.

This is not a system-design strategy. It is a trauma response.

You can't cure cancer "or whatever". If you want to build a computer system that does some thing, you actually need to hire experts in that thing, and have them work to both design and validate that the system is fit for the purpose of that thing.

Aside: But... are they, though?

I am not an oncologist; I do not particularly want to be writing about the specifics here, but, if I am going to make a claim like "you can't cure cancer this way" I need to back it up.

My first argument - and possibly my strongest - is that cancer is not cured.

QED.

But I guess, to Sam's credit, there is at least one other company partnering with OpenAI to do things that are specifically related to cancer. However, that company is still in a self-described "initial phase" and it's not entirely clear that it is going to work out very well.

Almost everything I can find about it online was from a PR push in the middle of last year, so it all reads like a press release. I can't easily find any independently-verified information.

A lot of AI hype is like this. A promising demo is delivered; claims are made that surely if the technology can solve this small part of the problem now, within 5 years surely it will be able to solve everything else as well!

But even the light-on-content puff-pieces tend to hedge quite a lot. For example, as the Wall Street Journal quoted one of the users initially testing it (emphasis mine):

The most promising use of AI in healthcare right now is automating "mundane" tasks like paperwork and physician note-taking, he said. The tendency for AI models to "hallucinate" and contain bias presents serious risks for using AI to replace doctors. Both Color's Laraki and OpenAI's Lightcap are adamant that doctors be involved in any clinical decisions.

I would probably not personally characterize "'mundane' tasks like paperwork and … note-taking" as "curing cancer". Maybe an oncologist could use some code I developed too; even if it helped them, I wouldn't be stealing valor from them on the curing-cancer part of their job.

Even fully giving it the benefit of the doubt that it works great, and improves patient outcomes significantly, this is medical back-office software. It is not super-intelligence.

It would not even matter if it were "super-intelligence", whatever that means, because "intelligence" is not how you do medical care or medical research. It's called "lab work" not "lab think".

To put a fine point on it: biomedical research fundamentally cannot be done entirely by reading papers or processing existing information. It cannot even be done by testing drugs in computer simulations.

Biological systems are enormously complex, and medical research on new therapies inherently requires careful, repeated empirical testing to validate the correspondence of existing research with reality. Not "an experiment", but a series of coordinated experiments that all test the same theoretical model. The data (which, in an LLM context, is "training data") might just be wrong; it may not reflect reality, and the only way to tell is to continuously verify it against reality.

Previous observations can be tainted by methodological errors, by data fraud, and by operational mistakes by practitioners. If there were a way to do verifiable development of new disease therapies without the extremely expensive ladder going from cell cultures to animal models to human trials, we would already be doing it, and "AI" would just be an improvement to efficiency of that process. But there is no way to do that and nothing about the technologies involved in LLMs is going to change that fact.

Knowing Things

The practice of science - indeed any practice of the collection of meaningful information - must be done by intentionally and carefully selecting inclusion criteria, methodically and repeatedly curating our data, building a model that operates according to rules we understand and can verify, and verifying the data itself with repeated tests against nature. We cannot just hoover up whatever information happens to be conveniently available with no human intervention and hope it resolves to a correct model of reality by accident. We need to look where the keys are, not where the light is.

Piling up more and more information in a haphazard and increasingly precarious pile will not allow us to climb to the top of that pile, all the way to heaven, so that we can attack and dethrone God.

Eventually, we'll just run out of disk space, and then lose the database file when the family gets a new computer anyway.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor! Special thanks also to Itamar Turner-Trauring and Thomas Grainger for pre-publication feedback on this article; any errors of course remain my own.

01 Apr 2025 12:47am GMT

Today on stream, I updated PINPal to fix the memorization algorithm.

If you haven't heard of PINPal before, it is a vault password memorization tool. For more detail on what that means, you can check it out the README, and why not give it a ⭐ while you're at it.

As I started writing up an update post I realized that I wanted to contextualize it a bit more, because it's a tool I really wish were more popular. It solves one of those small security problems that you can mostly ignore, right up until the point where it's a huge problem and it's too late to do anything about it.

In brief, PINPal helps you memorize new secure passcodes for things you actually have to remember and can't simply put into your password manager, like the password to your password manager, your PC user account login, your email account¹, or the PIN code to your phone or debit card.

Too often, even if you're properly using a good password manager for your passwords, you'll be protecting it with a password optimized for memorability, which is to say, one that isn't random and thus isn't secure. But I have also seen folks veer too far in the other direction, trying to make a really secure password that they then forget right after switching to a password manager. Forgetting your vault password can also be a really big deal, making you do password resets across every app you've loaded into it so far, so having an opportunity to practice it periodically is important.

PINPal uses spaced repetition to ensure that you remember the codes it generates.

While periodic forced password resets are a bad idea, if (and only if!) you can actually remember the new password, it is a good idea to get rid of old passwords eventually - like, let's say, when you get a new computer or phone. Doing so reduces the risk that a password stored somewhere on a very old hard drive or darkweb data dump is still floating around out there, forever haunting your current security posture. If you do a reset every 2 years or so, you know you've never got more than 2 years of history to worry about.

PINPal is also particularly secure in the way it incrementally generates your password; the computer you install it on only ever stores the entire password in memory when you type it in. It stores even the partial fragments that you are in the process of memorizing using the secure keyring module, avoiding plain-text whenever possible.

I've been using PINPal to generate and memorize new codes for a while, just in case², and the change I made today was because encountered a recurring problem. The problem was, I'd forget a token after it had been hidden, and there was never any going back. The moment that a token was hidden from the user, it was removed from storage, so you could never get a reminder. While I've successfully memorized about 10 different passwords with it so far, I've had to delete 3 or 4.

So, in the updated algorithm, the visual presentation now hides tokens in the prompt several memorizations before they're removed. Previously, if the password you were generating was 'hello world', you'd see hello world 5 times or so, times, then •••• world; if you ever got it wrong past that point, too bad, start over. Now, you'll see hello world, then °°°° world, then after you have gotten the prompt right without seeing the token a few times, you'll see •••• world after the backend has locked it in and it's properly erased from your computer.

If you get the prompt wrong, breaking your streak reveals the recently-hidden token until you get it right again. I also did a new release on that same livestream, so if this update sounds like it might make the memorization process more appealing, check it out via pip install pinpal today.

Right now this tool is still only extremely for a specific type of nerd - it's command-line only, and you probably need to hand-customize your shell prompt to invoke it periodically. But I'm working on making it more accessible to a broader audience. It's open source, of course, so you can feel free to contribute your own code!

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more things like it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

15 Jan 2025 12:54am GMT

Have you ever written some Python code that looks like this?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

from enum import Enum, auto

class SomeNumber(Enum):
    one = auto()
    two = auto()
    three = auto()

def behavior(number: SomeNumber) -> int:
    match number:
        case SomeNumber.one:
            print("one!")
            return 1
        case SomeNumber.two:
            print("two!")
            return 2
        case SomeNumber.three:
            print("three!")
            return 3

That is to say, have you written code that:

1. defined an enum with several members
2. associated custom behavior, or custom values, with each member of that enum,
3. needed one or more match / case statements (or, if you've been programming in Python for more than a few weeks, probably a big if/elif/elif/else tree) to do that association?

In this post, I'd like to submit that this is an antipattern; let's call it the "passive enum" antipattern.

For those of you having a generally positive experience organizing your discrete values with enums, it may seem odd to call this an "antipattern", so let me first make something clear: the path to a passive enum is going in the correct direction.

Typically - particularly in legacy code that predates Python 3.4 - one begins with a value that is a bare int constant, or maybe a str with some associated values sitting beside in a few global dicts.

Starting from there, collecting all of your values into an enum at all is a great first step. Having an explicit listing of all valid values and verifying against them is great.

But, it is a mistake to stop there. There are problems with passive enums, too:

1. The behavior can be defined somewhere far away from the data, making it difficult to:
   1. maintain an inventory of everywhere it's used,
   2. update all the consumers of the data when the list of enum values changes, and
   3. learn about the different usages as a consumer of the API
2. Logic may be defined procedurally (via if/elif or match) or declaratively (via e.g. a dict whose keys are your enum and whose values are the required associated value).
   1. If it's defined procedurally, it can be difficult to build tools to interrogate it, because you need to parse the AST of your Python program. So it can be difficult to build interactive tools that look at the associated data without just calling the relevant functions.
   2. If it's defined declaratively, it can be difficult for existing tools that do know how to interrogate ASTs (mypy, flake8, Pyright, ruff, et. al.) to make meaningful assertions about it. Does your linter know how to check that a dict whose keys should be every value of your enum is complete?

To refactor this, I would propose a further step towards organizing one's enum-oriented code: the active enum.

An active enum is one which contains all the logic associated with the first-party provider of the enum itself.

You may recognize this as a more generalized restatement of the object-oriented lens on the principle of "separation of concerns". The responsibilities of a class ought to be implemented as methods on that class, so that you can send messages to that class via method calls, and it's up to the class internally to implement things. Enums are no different.

More specifically, you might notice it as a riff on the Active Nothing pattern described in this excellent talk by Sandi Metz, and, yeah, it's the same thing.

The first refactoring that we can make is, thus, to mechanically move the method from an external function living anywhere, to a method on SomeNumber . At least like this, we present an API to consumers externally that shows that SomeNumber has a behavior method that can be invoked.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

from enum import Enum, auto

class SomeNumber(Enum):
    one = auto()
    two = auto()
    three = auto()

    def behavior(self) -> int:
        match self:
            case SomeNumber.one:
                print("one!")
                return 1
            case SomeNumber.two:
                print("two!")
                return 2
            case SomeNumber.three:
                print("three!")
                return 3

However, this still leaves us with a match statement that repeats all the values that we just defined, with no particular guarantee of completeness. To continue the refactoring, what we can do is change the value of the enum itself into a simple dataclass to structurally, by definition, contain all the fields we need:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

from dataclasses import dataclass
from enum import Enum
from typing import Callable

@dataclass(frozen=True)
class NumberValue:
    result: int
    effect: Callable[[], None]

class SomeNumber(Enum):
    one = NumberValue(1, lambda: print("one!"))
    two = NumberValue(2, lambda: print("two!"))
    three = NumberValue(3, lambda: print("three!"))

    def behavior(self) -> int:
        self.value.effect()
        return self.value.result

Here, we give SomeNumber members a value of NumberValue, a dataclass that requires a result: int and an effect: Callable to be constructed. Mypy will properly notice that if x is a SomeNumber, that x will have the type NumberValue and we will get proper type checking on its result (a static value) and effect (some associated behaviors)¹.

Note that the implementation of behavior method - still conveniently discoverable for callers, and with its signature unchanged - is now vastly simpler.

But what about...

Lookups?

You may be noticing that I have hand-waved over something important to many enum users, which is to say, by-value lookup. enum.auto will have generated int values for one, two, and three already, and by transforming those into NumberValue instances, I can no longer do SomeNumber(1).

For the simple, string-enum case, one where you might do class MyEnum: value = "value" so that you can do name lookups via MyEnum("value"), there's a simple solution: use square brackets instead of round ones. In this case, with no matching strings in sight, SomeNumber["one"] still works.

But, if we want to do integer lookups with our dataclass version here, there's a simple one-liner that will get them back for you; and, moreover, will let you do lookups on whatever attribute you want:

1

by_result = {each.value.result: each for each in SomeNumber}

enum.Flag?

You can do this with Flag more or less unchanged, but in the same way that you can't expect all your list[T] behaviors to be defined on T, the lack of a 1-to-1 correspondence between Flag instances and their values makes it more complex and out of scope for this pattern specifically.

3rd-party usage?

Sometimes an enum is defined in library L and used in application A, where L provides the data and A provides the behavior. If this is the case, then some amount of version shear is unavoidable; this is a situation where the data and behavior have different vendors, and this means that other means of abstraction are required to keep them in sync. Object-oriented modeling methods are for consolidating the responsibility for maintenance within a single vendor's scope of responsibility. Once you're not responsible for the entire model, you can't do the modeling over all of it, and that is perfectly normal and to be expected.

The goal of the Active Enum pattern is to avoid creating the additional complexity of that shear when it does not serve a purpose, not to ban it entirely.

A Case Study

I was inspired to make this post by a recent refactoring I did from a more obscure and magical² version of this pattern into the version that I am presenting here, but if I am going to call passive enums an "antipattern" I feel like it behooves me to point at an example outside of my own solo work.

So, for a more realistic example, let's consider a package that all Python developers will recognize from their day-to-day work, python-hearthstone, the Python library for parsing the data files associated with Blizzard's popular computerized collectible card game Hearthstone.

As I'm sure you already know, there are a lot of enums in this library, but for one small case study, let's look a few of the methods in hearthstone.enums.GameType.

GameType has already taken the "step 1" in the direction of an active enum, as I described above: as_bnet is an instancemethod on GameType itself, making it at least easy to see by looking at the class definition what operations it supports. However, in the implementation of that method (among many others) we can see the worst of both worlds:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

class GameType(IntEnum):
    def as_bnet(self, format: FormatType = FormatType.FT_STANDARD):
        if self == GameType.GT_RANKED:
            if format == FormatType.FT_WILD:
                return BnetGameType.BGT_RANKED_WILD
            elif format == FormatType.FT_STANDARD:
                return BnetGameType.BGT_RANKED_STANDARD
            # ...
            else:
                raise ValueError()
        # ...
        return {
            GameType.GT_UNKNOWN: BnetGameType.BGT_UNKNOWN,
            # ...
            GameType.GT_BATTLEGROUNDS_DUO_FRIENDLY: BnetGameType.BGT_BATTLEGROUNDS_DUO_FRIENDLY,
        }[self]

We have procedural code mixed with a data lookup table; raise ValueError mixed together with value returns. Overall, it looks like this might be hard to maintain this going forward, or to see what's going on without a comprehensive understanding of the game being modeled. Of course for most python programmers that understanding can be assumed, but, still.

If GameType were refactored in the manner above³, you'd be able to look at the member definition for GT_RANKED and see a mapping of FormatType to BnetGameType, or GT_BATTLEGROUNDS_DUO_FRIENDLY to see an unconditional value of BGT_BATTLEGROUNDS_DUO_FRIENDLY. Given that this enum has 40 elements, with several renamed or removed, it seems reasonable to expect that more will be added and removed as the game is developed.

Conclusion

If you have large enums that change over time, consider placing the responsibility for the behavior of the values alongside the values directly, and any logic for processing the values as methods of the enum. This will allow you to quickly validate that you have full coverage of any data that is required among all the different members of the enum, and it will allow API clients a convenient surface to discover the capabilities associated with that enum.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

10 Jan 2025 10:37pm GMT

Over the last decade, it has become a common experience to be using a social media app, and to perceive that app as saying something specific to you. This manifests in statements like "Twitter thinks Rudy Giuliani has lost his mind", "Facebook is up in arms about DEI", "Instagram is going crazy for this new water bottle", "BlueSky loves this bigoted substack", or "Mastodon can't stop talking about Linux". Sometimes this will even be expressed with "the Internet" as a metonym for the speaker's preferred social media: "the Internet thinks that Kate Middleton is missing".

However, even the smallest of these networks comprises literal millions of human beings, speaking dozens of different languages, many of whom never interact with each other at all. The hot takes that you see from a certain excitable sub-community, on your particular timeline or "for you" page, are not necessarily representative of "the Internet" - at this point, a group that represents a significant majority of the entire human population.

If I may coin a phrase, I will refer to these as "Diffuse, Amorphous, Nebulous, Generalized Internet Takes", or DANGITs, which handily evokes the frustrating feeling of arguing against them.

A DANGIT is not really a new "internet" phenomenon: it is a specific expression of the availability heuristic.

If we look at our device and see a bunch of comments in our inbox, particularly if those comments have high salience via being recent, emotive, and repeated, we will naturally think that this is what The Internet thinks. However, just because we will naturally think this does not mean that we will accurately think it.

It is worth keeping this concept in mind when participating in public discourse because it leads to a specific type of communication breakdown. If you are arguing with a DANGIT, you will feel like you are arguing with someone with incredibly inconsistent, hypocritical, and sometimes even totally self-contradictory views. But to be self-contradictory, one needs to have a self. And if you are arguing with 9 different people from 3 different ideological factions, all making completely different points and not even taking time to agree on the facts beforehand, of course it's going to sound like cacophonous nonsense. You're arguing with the cacophony, it's just presented to you in a way that deceives you into thinking that it's one group.

There are subtle variations on this breakdown; for example, it can also make people's taste seem incoherent. If it seems like one week the Interior Designer internet loves stark Scandinavian minimalism, and the next week baroque Rococo styles are making a comeback, it might seem like The Internet has no coherent sense of taste, and these things don't go together. That's because it doesn't! Why would you expect it to?

Most likely, you are simply seeing some posts from minimalists, and then, separately, some posts from Rococo aficionados. Any particular person's feed may be dedicated to a specific, internally coherent viewpoint, aesthetic, or ideology, but if you dump them all into a blender to separate them from their context, of course they will look jumbled together.

This is what social media does. It is context collapse as a service. Even if you eliminate engagement-maximizing algorithms and view everything perfectly chronologically, even if you have the world's best trust & safety team making sure that there is nothing harmful and no disinformation, social media - like email - inherently remains that context-collapsing blender. There's no way for it not to be; if two people you follow, who do not follow and are not aware of each other, are both posting unrelated things at the same time, you're going to see them at around the same time.

Do not argue with a DANGIT. Discussions are the internet are famously Pyrrhic battles to begin with, but if you argue with a DANGIT it's not that you will achieve a Pyrrhic victory, you cannot possibly achieve any victory, because you are shadowboxing an imagined consensus where none exits.

You can't win against something that isn't there.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more things like it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

16 Dec 2024 10:58pm GMT

Kamala Harris lost. Here we are. So it goes.

Are you sad? Are you scared?

I am very sad. I am very scared.

But, like everyone else in this position, most of all, I want to know what to do next.

A Mission For Progress

I believe that we should set up a missionary organization for progressive and liberal values.

In 2017, Kayla Chadwick wrote the now-classic article, "I Don't Know How To Explain To You That You Should Care About Other People". It resonated with millions of people, myself included. It expresses an exasperation with a populace that seems ignorant of economics, history, politics, and indeed unable to read the news. It is understandable to be frustrated with people who are exercising their electoral power callously and irresponsibly.

But I think in 2024, we need to reckon with the fact that we do, in fact, need to explain to a large swathe of the population that they should care about other people.

We had better figure out how to explain it soon.

Shared Values - A Basis for Hope

The first question that arises when we start considering outreach to the conservative-leaning or undecided independent population is, "are these people available to be convinced?".

To that, I must answer an unqualified "yes".

I know that some of you are already objecting. For those of us with an understanding of history and the mechanics of bigotry in the United States, it might initially seem like the answer is "no".

As the Nazis came to power in the 1920s, they were campaigning openly on a platform of antisemitic violence. Everyone knew what the debate was. It was hard to claim that you didn't, in spite of some breathtakingly cowardly contemporaneous journalism, they weren't fooling anyone.

It feels ridiculous to say this, but Hitler did not have support among Jews.

Yet, after campaigning on a platform of defaming immigrants, and Mexican immigrants specifically for a decade, a large part of what drove his victory is that Trump enjoyed a shockingly huge surge of support among the Hispanic population. Even some undocumented migrants - the ones most likely to be herded into concentration camps starting in January - are supporting him.

I believe that this is possible because, in order to maintain support of the multi-ethnic working-class coalition that Trump has built, the Republicans must maintain plausible deniability. They have to say "we are not racist", "we are not xenophobic". Incredibly, his supporters even say "I don't hate trans people" with startling regularity.

Most voters must continue to believe that hateful policies with devastating impacts are actually race-neutral, and are simply going to get rid of "bad" people. Even the ones motivated by racial resentment are mostly motivated by factually incorrect beliefs about racialized minorities receiving special treatment and resources which they are not in fact receiving.

They are victims of a disinformation machine. One that has rendered reality incomprehensible.

If you listen to conservative messaging, you can hear them referencing this all the time. Remember when JD Vance made that comment about Democrats calling Diet Mountain Dew racist?

Many publications wrote about this joke "bombing"¹, but the kernel of truth within it is this: understanding structural bigotry in the United States is difficult. When we progressives talk about it, people who don't understand it think that our explanations sound ridiculous and incoherent.

There's a reason that the real version of critical race theory is a graduate-level philosophy-of-law course, and not a couple of catch phrases.

If, without context, someone says that "municipal zoning laws are racist", this makes about as much sense as "Diet Mountain Dew is racist" to someone who doesn't already know what "redlining" is.

Conservatives prey upon this confusion to their benefit. But they prey on this because they must do so. They must do so because, despite everything, hate is not actually popular among the American electorate. Even now, they have to be deceived into it.

The good news is that all we need to do is stop the deception.

Politics Matter

If I have sold you on the idea that a substantial plurality of voters are available to be persuaded, the next question is: can we persuade them? Do we, as progressives, have the resources and means to do so? We did lose, after all, and it might seem like nothing we did had much of an impact.

Let's analyze that assumption.

Across the country, Trump's margins increased. However, in the swing states, where Harris spent money on campaigning, his margins increased less than elsewhere. At time of writing, we project that the safe-state margin shift will be 3.55% towards trump, and the swing-state margin shift will be 1.69%.

This margin was, sadly, too small for a victory, but it does show that the work mattered. Perhaps given more time, or more resources, it would have mattered just a little bit more, and that would have been decisive.

This is to say, in the places where campaign dollars were spent, even against the similar spending of the Trump campaign, we pushed the margin of support 1.86% higher within 107 days. So yes: campaigning matters. Which parts and how much are not straightforward, but it definitely matters.

This is a bit of a nonsensical comparison for a whole host of reasons², but just for a ballpark figure, if we kept this pressure up continuously during the next 4 years, we could increase support for a democratic candidate by 25%.

We Can Teach, Not Sell

Political junkies tend to overestimate the knowledge of the average voter. Even when we are trying to compensate for it, we tend to vastly overestimate how much the average voter knows about politics and policy. I suspect that you, dear reader, are a political junkie even if you don't think of yourself as one.

To give you a sense of what I mean, across the country, on Election day and the day after, there was a huge spike in interest for the Google query, "did Joe Biden drop out".

Consistently over the last decade, democratic policies are more popular than their opponents. Even deep red states, such as Kansas, often vote for policies supported by democrats and opposed by Republicans.

This confusion about policy is not organic; it is not voters' fault. It is because Republicans constantly lie.

All this ignorance might seem discouraging, but it presents an opportunity: people will not sign up to be persuaded, but people do like being informed. Rather than proselytizing via a hard sales pitch, it should be possible to offer to explain how policy connects to elections. And this is made so much the easier if so many of these folks already generally like our policies.

The Challenge Is Enormous

I've listed some reasons for optimism, but that does not mean that this will be easy.

Republicans have a tremendously powerful, decentralized media apparatus that reinforces their culture-war messaging all the time.

After some of the post-election analysis, "The Left Needs Its Own Joe Rogan" is on track to become a cliché within the week.³ While I am deeply sympathetic to that argument, the right-wing media's success is not organic; it is funded by petrochemical billionaires.

We cannot compete via billionaire financing, and as such, we have to have a way to introduce voters to progressive and liberal media. Which means more voters need social connections to liberals and progressives.

Good Works

The democratic presidential campaign alone spent a billion and a half dollars. And, as shown above, this can be persuasive, but it's just the persuasion itself.

Better than spending all this money on telling people what good stuff we would do for them if we were in power, we could just show them, by doing good stuff. We should live our values, not just endlessly reiterate them.

A billion dollars is a significant amount of power in its own right.

For historical precedent, consider the Black Panthers' Free Breakfast For Children program. This program absolutely scared the shit out of the conservative power structure, to the point that Nixon's FBI literally raided them for giving out free food to children.

Religious missionaries, who are famously annoying, often offset their annoying-ness by doing charitable work in the communities they are trying to reach. A lot of the country that we need to reach are religious people, and nominally both Christians and leftists share a concern for helping those in need, so we should find some cultural common ground there.

We can leverage that overlap in values by partnering with churches. This immediately makes such work culturally legible to many who we most need to reach.

Jobs Jobs Jobs

When I raised this idea with Philip James, he had been mulling over similar ideas for a long time, but with a slightly different tack: free career skills workshops from folks who are obviously "non-traditional" with respect to the average rural voter's cultural expectations. Recruit trans folks, black folks, women, and non-white immigrants from our tech networks.

Run the trainings over remote video conferencing to make volunteering more accessible. Run those workshops through churches as a distribution network.

There is good evidence that this sort of prolonged contact and direct exposure to outgroups, to help people see others as human beings, very effective politically.

However, job skills training is by no means the only benefit we could bring. There are lots of other services we could offer remotely, particularly with the skills that we in the tech community could offer. I offer this as an initial suggestion; if you have more ideas I'd love to hear them. I think the best ideas are ones where folks can opt in, things that feel like bettering oneself rather than receiving charity; nobody likes getting handouts, particularly from the outgroup, but getting help to improve your own skills feels more participatory.

I do think that free breakfast for children, specifically, might be something to start with because people are far more willing to accept gifts to benefit others (particularly their children, or the elderly!) rather than themselves.

Take Credit

Doing good works in the community isn't enough. We need to do visible good works. Attributable good works.

We don't want to be assholes about it, but we do want to make sure that these benefits are clearly labeled. We do not want to attach an obligation to any charitable project, but we do want to attach something to indicate where it came from.

I don't know what that "something" should be. The most important thing is that whatever "something" is appeals to set of partially-overlapping cultures that I am not really a part of - Midwestern, rural, southern, exurban, working class, "red state" - and thus, I would want to hear from people from those cultures about what works best.

But it's got to be something.

Maybe it's a little sticker, "brought to you by progressives and liberals. we care about you!". Maybe it's a subtle piece of consistent branding or graphic design, like a stylized blue stripe. Maybe we need to avoid the word "democrats", or even "progressive" or "liberal", and need some independent brand for such a thing, that is clearly tenuously connected but not directly; like the Coalition of Liberal and Leftist Helpful Neighbors or something.

Famously, when Trump sent everybody a check from the government, he put his name on it. Joe Biden did the same thing, and Democrats seem to think it's a good thing that he didn't take credit because it "wasn't about advancing politics", even though this obviously backfired. Republicans constantly take credit for the benefits of Democratic policies, which is one reason why voters don't know they're democratic policies.

Our broad left-liberal coalition is attempting to improve people's material conditions. Part of that is, and must be, advancing a political agenda. It's no good if we provide job trainings and free lunches to a community if that community is just going to be reduced to ruin by economically catastrophic tariffs and mass deportations.

We cannot do this work just for the credit, but getting credit is important.

Let's You And Me - Yes YOU - Get Started

I think this is a good idea, but I am not the right person to lead it.

For one thing, building this type of organization requires a lot of organizational and leadership skills that are not really my forte. Even the idea of filing the paperwork for a new 501(c)3 right now sounds like rolling Sisyphus's rock up the hill to me.

For another, we need folks who are connected to this culture, in ways that I am not. I would be happy to be involved - I do have some relevant technical skills to help with infrastructure, and I could always participate in some of the job-training stuff, and I can definitely donate a bit of money to a nonprofit, but I don't think I can be in charge.

You can definitely help too, and we will need a wide variety of skills to begin with, and it will definitely need money. Maybe you can help me figure out who should be in charge.

This project will be weaker without your support. Thus: I need to hear from you.

You can email me, or, if you'd prefer a more secure channel, feel free to reach out over Signal, where my introduction code is glyph.99 . Please start the message with "good works:" so I can easily identify conversations about this.

If I receive any interest at all, I plan to organize some form of meeting within the next 30 days to figure out concrete next steps.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more things like it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor! My aspirations for this support are more in the directions of software development than activism, but needs must, when the devil drives. Thanks especially to Philip James for both refining the idea and helping to edit this post, and to Marley Myrianthopoulos for assistance with the data analysis.

11 Nov 2024 4:01am GMT

It's the weekend, and I have some Thoughts about federated social media. So, buckle up, I guess, it's time to start some fights.

Recently there has been some discourse about Bluesky's latest fundraising round. I've been participating in conversations about this on Mastodon, and I think I might sometimes come across as a Mastodon partisan, but my feelings are complex and I really don't want to be boosting the ActivityPub Fediverse without qualification.

So here are some qualifications.

Bluesky Is Evil

To the extent that I am an ActivityPub partisan in the discourse between ActivityPub and ATProtocol, it is because I do not believe that Bluesky is a meaningfully decentralized social network. It is a social network, run by a company, which has a public API with some elements that might, one day, make it possible for it to be decentralized. But today, it is not, either practically or theoretically.

The Bluesky developers are putting in a ton of effort to maybe make it decentralized, hypothetically, someday. A lot of people think they will succeed. But ActivityPub (and, of course, Mastodon specifically) are already, today, meaningfully decentralized, as you can see on FediDB, there are instances with hundreds of thousands of people on them, before we even get to esoterica like the integrations Threads, Wordpress, Flipboard, and Ghost are doing.

The inciting incident for this post - that a lot of people are also angry about Bluesky raising millions of dollars from Evil Guys Doing Evil Stuff Capital - is indeed a serious concern. It lights the fuse that burns towards their eventual, inevitable incredible journey. ATProtocol is just an API, and that API will get shut off one day, whenever their funders get bored of the pretense of their network being "decentralized".

At time of writing, it is also interesting that 3 of the 4 times that the CEO of Bluesky has even skeeted the word "blockchain" is to say "no blockchain", to reassure users that the scam magnet of "Blockchain" is not actually near their product or protocol, which is a much harder position to maintain when your lead investor is "Blockchain Capital".

I think these are all valid criticisms of Bluesky. But I also think that the actual engineers working on the product are aware of these issues, and are making a significant effort to address them or mitigate them in any way they can. All that work can still be easily incinerated by a slow quarter in terms of user growth numbers or a missed revenue forecast when the VCs are getting impatient, but it's not nothing, it is a life's work.

Really, who among us could not have our life's ambitions trivially destroyed in an afternoon, simply because a billionaire decided that they should be? If you feel like you are safe from this, I have some bad news about how money works. So we are all doing our best in an imperfect system and maybe Bluesky is on to something here. That's eminently possible. They're certainly putting forth an earnest effort.

Mastodon Is Stupid

Meanwhile, not nearly as much has been made recently of Mastodon refusing funding from a variety of sources, when all indications are that funding is low, and plummeting, far below the level required to actually sustain the site, and they haven't done a financial transparency report for over a year, and that report was already nearly a year late.

Mastodon and the fediverse are not nearly in a position to claim moral superiority over Bluesky. Sure, taking blockchain VC money might seem like a rookie mistake, but going out of business because you are spurning every possible source of funding is not that wise either.

Some might think that, sure, Mastodon the company might die but at least the Fediverse as a whole will keep going strong, right? Lots of people run their own instances! I even find elements of this argument convincing, and I think there is probably some truth to it. But to really believe this argument as claimed, that it's a fait accompli that the fediverse will survive in some form, that all those self-run servers will be a robust network that will self-repair, requires believing some obviously false stuff. It is frankly unprofitable to run a Fediverse instance. Realistically, if you want to operate a mastodon server for yourself, it is going to cost at least $100/year once you include stuff like having a domain name, and managing the infrastructure costs is a complex problem that keeps getting harder to manage as the software itself gets slower.

Cory Doctorow has recently argued that this is all worth it, because at least on Mastodon, you're in control, not at the whims of centralized website operators like Bluesky. In his words,

On Mastodon (and other services based on Activitypub), you can easily leave one server and go to another, and everyone you follow and everyone who follows you will move over to the new server. If the person who runs your server turns out to be imperfect in a way that you can't endure, you can find another server, spend five minutes moving your account over, and you're back up and running on the new server

He concludes:

Any system where users can leave without pain is a system whose owners have high switching costs and whose users have none

(Emphasis mine).

This is a beautiful vision. It is, however, an incorrect assessment of the state of the Fediverse as it stands today. It's not true in two important ways:

First, if you look at any account of a user's fediverse account migration, like this one from Steve Bate or this one from the Ente project or this one from Erin Kissane, you will see that it is "painful for the foreseeable future" or "wasn't as seamless as advertised", and that "the best time to […] migrate instances […] is never". This language does not presage a pleasant experience, as Doctorow puts it, "without pain".

Second, migration is an active process that requires engagement from the instance that hosts you. If you have been blocked or banned, or had your account terminated, you are just out of luck. You do not have control over your data or agency over your online identity unless you've shelled out the relatively exorbitant amount of money to actually operate your own instance.

In short, ActivityPub is no panacea. A federated system is not really a "decentralized" system, as much as it is a bunch of smaller centralized systems that all talk to each other. You still need to know, and care, about your social and financial relationship to the operators of your instance. There is probably no getting away from this, like, just generally on the Internet, no matter how much peer-to-peer software we deploy, but there certainly isn't in the incomplete mess that is ActivityPub.

JOIN, or DIE.

Neither Mastodon (or ActivityPub) nor Bluesky (or ATProtocol) has a comprehensive solution to the problem of decentralized social media. These companies, and these protocols, are both deeply flawed and if everything keeps bumping along as it is, I believe both are likely to fail. At different times, on different timelines, and for different reasons, but fail nonetheless.

However, these networks are both small and growing, and we are not yet in the phase of enshittification where margins are shrinking and audiences are captured and the screws must be tightened to juice revenue. There are stil possibilities. Mastodon is crowdfunded and what they lack in resources they make up for in flexibility and scrappiness. Bluesky has money and while there will eventually be a need to monetize somehow, they have plenty of runway to come up with that answer, and a lot of sophisticated protocol work has been done. Not enough to make a complete circut and allow users true, practical decentralization, but it's not nothing, either.

Mastodon and Bluesky are both organizations with humans in them, and piles of data that is roughly schema-compatible even if the nuances and details are different. I know that there is a compatible model becuse thanks to both platforms being relatively open, there is a functioning ActivityPub/ATProtocol bridge in the form of Brid.gy Fed. You can use it today, and I highly recommend that you do so, so that "choice of protocol" does not fully define your audience. If you're on bluesky, follow this account, and if you're on Mastodon or elsewhere on the Fediverse, search for and follow @bsky.brid.gy@bsky.brid.gy.

The reality that fans of decentralized, independent social media must confront is that we are a tiny audicence right now. Whichever site we are looking at, we are talking about a few million monthly active users at best, in a world where even the pathetic husk of Twitter still has hundreds of millions and Facebook has billions. Interneceine fights are not going to get us anywhere. We need to build bridges and links and connect our networks as densely as possible. If I'm being honest, Bridgy Fed looks like a pretty janky solution, but it's something, and we need to start doing something soon, so we do not collectively become a permanent minority that mass markets can safely ignore.

As users, we need to set an example, so that the developers of the respective platforms get their shit together and work together directly so that workarounds like Bridgy are not required. Frankly, this is mostly on the ActivityPub and Mastodon devs, as far as I can tell. Unfortunately, not a lot of this seems to be public, or at least I haven't witnessed a lot of it directly, but I have heard repeatedly that the ActivityPub developers are prickly, and this is one high-profile public example where an ActivityPub partisan is incredibly, pointlessly hostile and borderline harrassing towards someone - Mike Masnick, a long-time staunch advocate for open protocols and open patents, someone with a Mastodon account, and thus as good a prospective ally as the ActivityPub fediverse might reasonably find - explaining some of the relative benefits of Bluesky.

Most of us are technology nerds in one way or another. In that way we can look at signifiers like "ActivityPub" and "ATProtocol", and feel like these are hard boundaries around different all-encompassing structures for the future, and thus tribes we must join and support.

A better way to look at this, however, is to see social entities like Mastodon gGmbH and Bluesky PBC - or, more to the point, Fosstodon, SFBA Social, Hachyderm (and maybe, one day, even an instance which isn't fully just for software development nerds), as groups that deploy these protocols to access some data that they publish, just as they might publish their website over HTTP or their newsletters over SMTP. There are technical challenges involved in bridging between mutually unintelligible domain models, but that is, like, network software's whole deal. Most software is just some kind of translation from one format or context to another. The best possible future for the fediverse is the one where users care as much about the distinction between ATProtocol and ActivityPub as they do about the distinction between POP3 and IMAP.

To both developers and users of these systems, I say: get it together. Be nice to each other. Because the rest of the social media ecosystem is sure as shit not going to be nice to us if we ever see even a hint of success and start to actually cut into their user base.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

03 Nov 2024 9:49pm GMT

Starting with 0.3.0, Astral's uv brought many great features, including support for cross-platform lock files uv.lock. Together with subsequent fixes, it has become Python's finest workflow tool for my (non-scientific) use cases. Here's how I build production-ready containers, as fast as possible.

24 Sep 2024 12:00am GMT

One of my first TIL entries was about how you can imitate Node's node_modules semantics in Python on UNIX-like operating systems. A lot has happened since then (to the better!) and it's time for an update. direnv still rocks, though.

23 Sep 2024 12:00am GMT

When you build Python, you can pass various options to ./configure that change aspects of how it is built. There is documentation for all of these options, and they are things like --prefix to tell the build where to install itself, --without-pymalloc if you have some esoteric need for everything to go through a custom memory allocator, or --with-pydebug.

One of these options only matters on macOS, and its effects are generally poorly understood. The official documentation just says "Create a Python.framework rather than a traditional Unix install." But… do you need a Python.framework? If you're used to running Python on Linux, then a "traditional Unix install" might sound pretty good; more consistent with what you are used to.

If you use a non-Framework build, most stuff seems to work, so why should anyone care? I have mentioned it as a detail in my previous post about Python on macOS, but even I didn't really explain why you'd want it, just that it was generally desirable.

The traditional answer to this question is that you need a Framework build "if you want to use a GUI", but this is demonstrably not true. At first it might not seem so, since the go-to Python GUI test is "run IDLE"; many non-Framework builds also omit Tkinter because they don't ship a Tk dependency, so IDLE won't start. But other GUI libraries work fine. For example, uv tool install runsnakerun / runsnake will happily pop open a GUI window, Framework build or not. So it bears some explaining

Wait, what is a "Framework" anyway?

Let's back up and review an important detail of the mac platform.

On macOS, GUI applications are not just an executable file, they are organized into a bundle, which is a directory with a particular layout, that includes metadata, that launches an executable. A thing that, on Linux, might live in a combination of /bin/foo for its executable and /share/foo/ for its associated data files, is instead on macOS bundled together into Foo.app, and those components live in specified locations within that directory.

A framework is also a bundle, but one that contains a library. Since they are directories, Applications can contain their own Frameworks and Frameworks can contain helper Applications. If /Applications is roughly equivalent to the Unix /bin, then /Library/Frameworks is roughly equivalent to the Unix /lib.

App bundles are contained in a directory with a .app suffix, and frameworks are a directory with a .framework suffix.

So what do you need a Framework for in Python?

The truth about Framework builds is that there is not really one specific thing that you can point to that works or doesn't work, where you "need" or "don't need" a Framework build. I was not able to quickly construct an example that trivially fails in a non-framework context for this post, but I didn't try that many different things, and there are a lot of different things that might fail.

The biggest issue is not actually the Python.framework itself. The metadata on the framework is not used for much outside of a build or linker context. However, Python's Framework builds also ship with a stub application bundle, which places your Python process into a normal application(-ish) execution context all the time, which allows for various platform APIs like [NSBundle mainBundle] to behave in the normal, predictable ways that all of the numerous, various frameworks included on Apple platforms expect.

Various Apple platform features might want to ask a process questions like "what is your unique bundle identifier?" or "what entitlements are you authorized to access" and even beginning to answer those questions requires information stored in the application's bundle.

Python does not ship with a wrapper around the core macOS "cocoa" API itself, but we can use pyobjc to interrogate this. After installing pyobjc-framework-cocoa, I can do this

1
2

>>> import AppKit
>>> AppKit.NSBundle.mainBundle()

On a non-Framework build, it might look like this:

1

NSBundle </Users/glyph/example/.venv/bin> (loaded)

But on a Framework build (even in a venv in a similar location), it might look like this:

1

NSBundle </Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app> (loaded)

This is why, at various points in the past, GUI access required a framework build, since connections to the window server would just be rejected for Unix-style executables. But that was an annoying restriction, so it was removed at some point, or at least, the behavior was changed. As far as I can tell, this change was not documented. But other things like user notifications or geolocation might need to identity an application for preferences or permissions purposes, respectively. Even something as basic as "what is your app icon" for what to show in alert dialogs is information contained in the bundle. So if you use a library that wants to make use of any of these features, it might work, or it might behave oddly, or it might silently fail in an undocumented way.

This might seem like undocumented, unnecessary cruft, but it is that way because it's just basic stuff the platform expects to be there for a lot of different features of the platform.

/etc/ builds

Still, this might seem like a strangely vague description of this feature, so it might be helpful to examine it by a metaphor to something you are more familiar with. If you're familiar with more Unix style application development, consider a junior developer - let's call him Jim - asking you if they should use an "/etc build" or not as a basis for their Docker containers.

What is an "/etc build"? Well, base images like ubuntu come with a bunch of files in /etc, and Jim just doesn't see the point of any of them, so he likes to delete everything in /etc just to make things simpler. It seems to work so far. More experienced Unix engineers that he has asked react negatively and make a face when he tells them this, and seem to think that things will break. But their app seems to work fine, and none of these engineers can demonstrate some simple function breaking, so what's the problem?

Off the top of your head, can you list all the features that all the files that /etc is needed for? Why not? Jim thinks it's weird that all this stuff is undocumented, and it must just be unnecessary cruft.

If Jim were to come back to you later with a problem like "it seems like hostname resolution doesn't work sometimes" or "ls says all my files are owned by 1001 rather than the user name I specified in my Dockerfile" you'd probably say "please, put /etc back, I don't know exactly what file you need but lots of things just expect it to be there".

This is what a framework vs. a non-Framework build is like. A Framework build just includes all the pieces of the build that the macOS platform expects to be there. What pieces do what features need? It depends. It changes over time. And the stub that Python's Framework builds include may not be sufficient for some more esoteric stuff anyway. For example, if you want to use a feature that needs a bundle that has been signed with custom entitlements to access something specific, like the virtualization API, you might need to build your own app bundle. To extend our analogy with Jim, the fact that /etc exists and has the default files in it won't always be sufficient; sometimes you have to add more files to /etc, with quite specific contents, for some features to work properly. But "don't get rid of /etc (or your application bundle)" is pretty good advice.

Do you ever want a non-Framework build?

macOS does have a Unix subsystem, and many Unix-y things work, for Unix-y tasks. If you are developing a web application that mostly runs on Linux anyway and never care about using any features that touch the macOS-specific parts of your mac, then you probably don't have to care all that much about Framework builds. You're not going to be surprised one day by non-framework builds suddenly being unable to use some basic Unix facility like sockets or files. As long as you are aware of these limitations, it's fine to install non-Framework builds. I have a dozen or so Pythons on my computer at any given time, and many of them are not Framework builds.

Framework builds do have some small drawbacks. They tend to be larger, they can be a bit more annoying to relocate, they typically want to live in a location like /Library or ~/Library. You can move Python.framework into an application bundle according to certain rules, as any bundling tool for macOS will have to do, but it might not work in random filesystem locations. This may make managing really large number of Python versions more annoying.

Most of all, the main reason to use a non-Framework build is if you are building a tool that manages a fleet of Python installations to perform some automation that needs to know about Python installs, and you want to write one simple tool that does stuff on Linux and on macOS. If you know you don't need any platform-specific features, don't want to spend the (not insignificant!) effort to cover those edge cases, and you get a lot of value from that level of consistency (for example, a teaching environment or interdisciplinary development team with a lot of platform diversity) then a non-framework build might be a better option.

Why do I care?

Personally, I think it's important for Framework builds to be the default for most users, because I think that as much stuff should work out of the box as possible. Any user who sees a neat library that lets them get control of some chunk of data stored on their mac - map data, health data, game center high scores, whatever it is - should be empowered to call into those APIs and deal with that data for themselves.

Apple already makes it hard enough with their thicket of code-signing and notarization requirements for distributing software, aggressive privacy restrictions which prevents API access to some of this data in the first place, all these weird Unix-but-not-Unix filesystem layout idioms, sandboxing that restricts access to various features, and the use of esoteric abstractions like mach ports for communications behind the scenes. We don't need to make it even harder by making the way that you install your Python be a surprise gotcha variable that determines whether or not you can use an API like "show me a user notification when my data analysis is done" or "don't do a power-hungry data analysis when I'm on battery power", especially if it kinda-sorta works most of the time, but only fails on certain patch-releases of certain versions of the operating system, becuase an implementation detail of a proprietary framework changed in the meanwhile to require an application bundle where it didn't before, or vice versa.

More generally, I think that we should care about empowering users with local computation and platform access on all platforms, Linux and Windows included. This just happens to be one particular quirk of how native platform integration works on macOS specifically.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. For this one, thanks especially to long-time patron Hynek who requested it specifically. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor! I am also available for consulting work if you think your organization could benefit from expertise on topics like "how can we set up our Mac developers' laptops with Python".

11 Sep 2024 7:43pm GMT

Codecov's unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn't rely on third-party services.

03 Sep 2024 12:00am GMT

Whenever I publish something about my Python Docker workflows, I invariably get challenged about whether it makes sense to use virtual environments in Docker containers. As always, it's a trade-off, and I err on the side of standards and predictability.

02 Sep 2024 11:00am GMT

If a high-status member of a community that you participate in is accused of misbehavior, you may want to defend them. You may even write a long essay in their defense.

In that essay, it may seem only natural to begin with a lengthy enumeration of the accused's positive personal qualities. To extol the quality of their career and their contributions to your community. To talk about how nice they are. To be a character witness in the court of public opinion.

If you do this, you are not defending them. You are proving the point. This is exactly how missing stairs come to exist. People don't get away with bad behavior if they don't have high status and a good reputation already.

Sometimes, someone with antisocial inclinations seeks out status, in order to facilitate their bad behavior. Sometimes, a good, but, flawed person does a lot of really good work and thereby accidentally ends up with more status than they were expecting to have, and they don't know how to handle it. In either case, bad behavior may ensue.

If you truly believe that your fave is being accused or punished unjustly, focus on the facts. What, specifically, has been alleged? How are these allegations substantiated? What verifiable evidence exists to the contrary? If you feel that someone is falsely accusing them to ruin their reputation, is there evidence to support your claim that the accusation is false? Ask yourself the question: what information do you have, that is leading to your correct analysis of the situation, that the people making the accusations do not have, which might be leading them into error?

But, also, maybe just… don't?

The urge to defend someone like this is much more likely to come from a sense of personal grievance than justice. Consider: does it feel like you are being attacked, when your fave has been attacked? Is there a tightness in your chest, heat rising on your cheeks? Do you feel suddenly defensive?

Do you think that defensiveness is likely to lead to you making good, rational decisions about what steps to take next?

Let your heroes face accountability. If they are really worth your admiration, they might accept responsibility and make amends. Or they might fight the accusations with their own real evidence - evidence that you, someone peripheral to their situation, are unlikely to have - and prove the accusations wrong.

They might not want your defense. Even if they feel like they do want it in the moment - they are human too, after all, and facing accountability does not feel good to us humans - is the intensified feeling that they can't let down their supporters who believe in them likely to make them feel less defensive and panicked?

In either case, your character defense is unlikely to serve them. At best it helps them stay on an ego trip, at worst it muddies the waters and might confuse the collection of facts that would, if considered dispassionately, properly exonerate them.

Do you think that I am pretending to speak in generalities but really talking about one specific recent event?

Wrong!

Just in this last week, I have read 2 different blog posts about 2 completely different people in completely unrelated communities and both of their authors need to read this. But each of those were already of a type, one that I've read dozens of instances of in the past.

It is a very human impulse to perceive a threat to someone we think well of, and to try to defend against that threat. But the consequences of someone's own actions are not a threat you can defend them from.

16 Aug 2024 7:53pm GMT

Updated 2024-07-04: After some discussion, added an epilogue going into more detail about the value of the distinction between the two types of tokens.

In 2015, Dan McKinley laid out a model for software teams selecting technologies. He proposed that each team have a limited supply of "innovation tokens", and, when selecting a technology, they can choose boring ones for free but "innovative" ones cost a token. This implies that we all know which technologies are innovative, and we assume that they are inherently costly, so we want to restrict their supply.

That model has become popular to the point that it is now part of the vernacular. In many discussions, it is accepted as received wisdom, or even common sense.

In this post I aim to show you that despite being superficially helpful, this model is wrong, and in fact, may be counterproductive. I believe it is an attractive nuisance in computer programming discourse.

In fairness to Mr. McKinley, the model he described in this post is:

1. nearly a decade old at this point, and
2. much more nuanced in its description of the problem with "innovation" than the subsequent memetic mutation of the concept.

While I will be referencing McKinley's post, and I do take some issue with it, I am reacting more strongly to the life of its own that this idea has taken on once it escaped its original context. There are a zillion worse posts rehashing this concept, on blogs and LinkedIn, but I won't be linking to them because the goal is not to call anybody out.

To some extent I am re-raising McKinley's own caveats and reinforcing them. So I may be arguing with a strawman, but it's a strawman I have seen deployed with some regularity over the years.

To reduce it to its core, this strawman is "don't use new or interesting technology, and if you have to, only use a little bit".

Within the broader culture of programmers, an "innovation token" has become a shorthand to smear any technology perceived - almost always based on vibes, not data - as risky, and the adoption of novel approaches as pretentious and unserious. Speaking of programmer culture though, I do have to acknowledge there is also a pervasive tendency for us to get distracted by novelty and waste time on puzzles rather than problem-solving, so I understand where the reactionary attitude represented by the concept of an innovation token comes from.

But it is reactionary.

At its worst, it borders on anti-intellectualism. I have heard it used on more than one occasion as a thought-terminating cliche to discard a potentially promising new tool. But before I get into that, let me try to give a sympathetic summary of the idea, because the model is not entirely bad.

It has been popular for a long time because it does work okay as an heuristic.

The real problem that McKinley is describing is operational overhead. When programmers make a technology selection, we are often considering how difficult it will make the programming. Innovative technology selections are, by definition, less mature.

That lack of maturity - particularly in the open source world - often means that the project is in a part of its lifecycle where it is concerned with development affordances more than operational ones. Therefore, the stereotypical innovative project, even one which might legitimately be a big improvement to development velocity, will create more operational overhead. That operational overhead creates a hidden cost for the operations team later on.

This is a point I emphatically agree with. When selecting a technology, you should consider its ease of operation more than its ease of development. If your team is successful, they will be operating and maintaining it far longer than they are initially integrating and deploying it.

Furthermore, some operational overhead is inevitable. You will need to hire people to mitigate it. More popular, more mature projects will have a bigger talent pool to hire from, so your training costs will be lower, and those training costs are part of your operational cost too.

Rationing innovation tokens therefore can work as a reasonable heuristic, or proxy metric, for avoiding a mess of complex operational problems associated with dependencies that are expensive to operate and hard to hire for.

There are some minor issues I want to point out before getting to the overarching one.

1. "has a lot of operational overhead" is a stereotype of a new technology, not an inherent property. If you want to reject a technology on the basis of being too high-overhead, at least look into its actual overhead a little bit. Sometimes, especially in 2024 as opposed to 2015, the point of a new, shiny piece of tech is to address operational issues that the more boring, older one had.
2. "hard to learn" is also a stereotype; if "newer" meant "harder" then we would all be using troff rather than Google Docs. Actually ask if the innovativeness is making things harder or easier; don't assume.
3. You are going to have to train people on your stack no matter what. If a technology is adding a lot of value, it's absolutely worth hiring for general ability and making a plan to teach people about it. You are going to have to do this with the core technology of your product anyway.

As I said, though, these are minor issues. The big problem with modeling operational overhead as an "innovation token" is that an even bigger concern than selecting an innovative tool is selecting too many tools.

The impulse to select more tools and make your operational environment more complex can be made worse by trying to avoid innovative tools. The important thing is not "less innovation", but more consistency. To illustrate this, let's do a simple thought experiment.

Let's say you're going to make a web app. There's a tool in Haskell that you really like for a critical part of your app's problem domain. You don't want to spend more than one innovation token though, and everything in Haskell is inherently innovative, so you write a little service that just does that one part and you write the rest of your app in Ruby, calling into that service whenever you need to use that thing. This will appropriately restrict your "innovation token" expenditure.

Does doing this actually reduce your operational overhead, though?

First, you will have to find a team that likes both Ruby and Haskell and sees no problem using both. If you are not familiar with the cultural proclivities of these languages, suffice it to say that this is unlikely. Hiring for Haskell programmers is hard because there are fewer of them than Ruby programmers, but hiring for polyglot Haskell/Ruby programmers who are happy to do either is going to be really hard.

Since you will need to find different people to write in the different languages, even in the best case scenario, you will have two teams: the Haskell team and the Ruby team. Even if you are incredibly disciplined about inter-service responsibilities, there will be some areas where duplication of code is necessary across those services. Disagreements will arise and every one of these disagreements will be a source of social friction and software defects.

Then, you need to set up separate CI pipelines for each language, separate deployment systems, and of course, separate databases. Right away you are effectively doubling your workload.

In the worse, and unfortunately more likely scenario, there will be enormous infighting between these two teams. Operational incidents will be more difficult to manage because rather than learning the Haskell tools for operational visibility and disseminating that institutional knowledge amongst your team, you will be half-learning the lessons from two separate ecosystems and attempting to integrate them. Every on-call engineer will be frantically trying to learn a language ecosystem they don't use regularly, or you will double the size of your on-call rotation. The Ruby team may start to resent the Haskell team for getting to exclusively work on the fun parts of the problem while they are doing things that look more like rote grunt work.

A better way to think about the problem of managing operational overhead is, rather than "innovation tokens", consider "boundary tokens".

That is to say, rather than evaluating the general sense of weird vibes from your architecture, consider the consistency of that architecture. If you're using Haskell, use Haskell. You should be all-in on Haskell web frameworks, Haskell ORMs, Haskell OAuth integrations, and so on.¹ To cross the boundary out of Haskell, you need to spend a boundary token, and you shouldn't have many of those.

I submit that the increased operational overhead that you might experience with an all-Haskell tool selection will be dwarfed by the savings that you get by having a team that is aligned with each other, that can communicate easily, and that can share programs with each other without needing to first strategize about a channel for the two pieces of work to establish bidirectional communication. The ability to simply call a function when you need to call it is very powerful, and extremely underrated.

Consistency ought to apply at each layer of the stack; it is perhaps most obvious with programming languages, but it is true of web frameworks, test frameworks, cryptographic libraries, you name it. Make a choice and stick with it, because every deviation from that choice carries a significant cost. Moreover this cost is a hidden cost, in the same way that the operational downsides of an "innovative" tool that hasn't seen much production use might be hidden.

Discarding a more standard tool in favor of a tool more consistent with your architecture extends even to fairly uncontroversial, ubiquitous tools. For example, one of my favorite architectural patterns is to forego the use of the venerable - and very boring - Cron, the UNIX task-scheduler. Instead of Cron, it can make a lot of sense to have hand-written bespoke code for scheduling tasks within the application. Within the "innovation tokens" model, this is a very silly waste of a token!

Just use Cron! Everybody knows how to use Cron!

Except… does everybody know how to use Cron? Here are some questions to consider, if you're about to roll out a big dependency on Cron:

1. How do you write a unit test for a scheduling rule with Cron?
2. Can you even remember how to write a cron rule that runs at the times you want?
3. How do you inject secrets and configuration variables into the distinct and somewhat idiosyncratic runtime execution environment of Cron?
4. How do you know that you did that variable-injection properly until the job actually runs, possibly in the middle of the night?
5. How do you deploy your monitoring and error-logging frameworks to observe your scripts run under Cron?

Granted, this architectural choice is less controversial than it once was. Cron used to be ambiently available on whatever servers you happened to be running. As container-based deployments have increased in popularity, this sense that Cron is just kinda around has gone away, and if you need to run a container that just runs Cron, much of the jankiness of its deployment is a lot more immediately visible.

There is friction at the boundary between things. That friction is a cost, but sometimes it's a cost worth paying.

If there's a really good library in Haskell and a really good library in Ruby and you really do want to use them both, maybe it makes sense to actually have multiple services. As your team gets larger and more mature, the need to bring in more tools, and the ability to handle the associated overhead, will only increase over time. But the place that the cost comes in the most is at the boundary between tools, not in the operational deficiencies of any one particular tool.

Even in a bog-standard web application with the most boring, least innovative tech stack imaginable (PHP, MySQL, HTML, CSS, JavaScript), many of the annoying points of friction are where different, inconsistent technologies make contact. If you are a programmer working on the web yourself, consider your own impression of the level of controversy of these technologies:

• CSS frameworks that attempt to bridge the gap between CSS and JavaScript, like Bootstrap or Tailwind.
• ORMs that attempt to bridge the gap between SQL and your backend language of choice
• RPC systems that attempt to connect disparate services together using simple abstractions.

Consider that there are far more complex technical tools in terms of required skills to implement them, like computer vision or physics simulation, tools which are also pretty widely used, which consistently generate lower levels of controversy. People do have strong feelings about these things as well, of course, and it's hard to find things to link to that show "this isn't controversial", but, like, search your feelings, you know it to be true.

You can see the benefits of the boundary token approach in programming language design. Many of the most influential and best-loved programming languages had an impact not by bundling together lots of tools, but by making everything into one thing:

• LISP: everything is a list
• Smalltalk: everything is an object
• ML: everything is an algebraic data type
• Forth: everything is a stack

There is a tremendous power in thinking about everything as a single kind of thing, because then you don't have to juggle lots of different ideas about different kinds of things; you can just think about your problem.

When people complain about programming languages, they're often complaining about how many different kinds of thing they have to remember in order to use it.

If you keep your boundary-token budget small, and allow your developers to accomplish as much as possible while staying within a solution space delineated by a single, clean cognitive boundary, I promise you can innovate as much as you want and your operational costs will remain manageable.

Epilogue

In subsequent Mastodon discussion of this post on with Matt Campbell and Meejah, I realized that I may not have made it entirely clear why I feel the distinction between "boundary" and "innovation" tokens is important. I do say above that the "innovation token" model can be a useful heuristic, so why bother with a new, but slightly different heuristic? Especially since most experienced engineers - indeed, McKinley himself - would budget "innovation" quite similarly to "boundaries", and might even consider the use of more "innovative" Haskell tools in my hypothetical scenario to not even be an expenditure of innovation tokens at all.

To answer that, I need to highlight the purpose of having heuristics like this in the first place. These are vague, nebulous guidelines, not hard and fast rules. I cannot give you a token calculator to plug your technical decisions into. The purpose of either token heuristic is to facilitate discussions among a team.

With a team of skilled and experienced engineers, the distinction is meaningless. Senior and staff engineers (at least, the ones who deserve their level) will intuit the goals behind "innovation tokens" and inherently consider things like operational overhead anyway. In practice, a high-performing, well-aligned team discussing innovation tokens and one discussing boundary tokens will look functionally indistinguishable.

The distinction starts to be important when you have management pressures, nervous executives, inexperienced engineers, a fresh team without existing consensus about core technology choices, and so on. That is to say, most teams that exist in the messy, perpetually in medias res world of the software industry.

If you are just getting started on a project and you have a bunch of competent but disagreeable engineers, the words "innovation" and "boundaries" function very differently.

If you ask, "is this an innovation" about a particular technical tool, you are asking your interlocutor to pull in a bunch of their skills and experience to subjectively evaluate the relative industry-wide, or maybe company-wide, or maybe team-wide² newness of the thing being discussed. The discussion of whether it counts as boring or innovative is immediately fraught with a ton of subjective, difficult-to-quantify information about costs of hiring, difficulty of learning, and your impression of the feelings of hundreds or thousands of people outside of your team. And, yes, ultimately you do need to have an estimate of all that stuff, but starting your "is it OK to use this" conversation by simultaneously arguing about all those subjective judgments is setting yourself up for failure.

Instead, if you ask "does this introduce a boundary between two different technologies with different conceptual models", while that is not a perfectly objective question, it is much easier for your team to answer, with much crisper intermediary factual questions. What are the two technologies? What are the models? How much do they differ? You can just hash out the answers to each one within the team directly, rather than needing to sift through the last few years of Stack Overflow developer surveys to determine relative adoption or popularity of technologies in the world at large.

Restricting your supply of either boundary or innovation tokens is a good idea, but achieving unanimity within your team about what your boundaries are is always going to be easier than deciding what your innovations are.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor! I am also available for consulting work if you think your organization could benefit from expertise on topics like "how can we make our architecture more consistent".

04 Jul 2024 7:54pm GMT