Since introducing wrapture at the end of August I have been putting out a post every day or so, first working through how it is used in unit tests and then how the same bindings trace a running application. With the last of the tracing posts now done, this is the point where I wanted to stop, take stock, and pull everything together in one place for anyone who wants to try it out.

Where things stand

The short version is that wrapture has been bumped to version 1.0.0b1. That move from alpha to beta is deliberate. I am happy with the APIs as they are and I am not seeing a need to change them, so what the beta series needs now is not more features from me but people using it on real code and reporting back. Reports of it working, or not working, on something I never thought to test, and of what confused you or what you found missing, are what will decide whether anything changes before a release candidate. They go to the issue tracker on GitHub.

One area where feedback would be especially useful is the OpenTelemetry export. The events wrapture records are mapped through to spans, attributes and metrics, and the intent was for that mapping to follow the OpenTelemetry semantic conventions. As I explained in the introductory post, the code was written by an AI under my direction rather than by me, so I cannot claim to have checked every attribute against the conventions myself, and can only say I hope the information is being mapped through okay. If you spend your days looking at traces in an OpenTelemetry backend and find that what arrives is not named or shaped the way the conventions say it should be, or the way your backend expects, I would like to hear about it while it is still cheap to change.

The companion instrumentation packages have been bumped to 1.0.0b1 at the same time. Until 1.0.0 is final a plain pip install wrapture picks up the latest pre-release automatically, so there is no need to pin a specific version to try it.

The links you need are:

• wrapture on GitHub, and the documentation on ReadTheDocs. The getting started page is the place to begin, and if you are coming from unittest.mock there is a comparison page mapping each mock idiom to its wrapture counterpart.
• wrapture-instrumentation, the core collection of packaged instrumentation. It covers the web frameworks Flask, Django, FastAPI, Starlette and aiohttp, the servers uvicorn, werkzeug and wsgiref, the HTTP clients requests, httpx, urllib3, aiohttp, urllib.request and http.client, XML-RPC on both ends, gRPC, SQLAlchemy and sqlite3, and Jinja2 templates.
• wrapture-instrumentation-aws, for the AWS SDK (boto3 and botocore), with every AWS API call recorded as one event.
• wrapture-instrumentation-postgresql, for the PostgreSQL drivers psycopg, psycopg2 and asyncpg.
• wrapture-instrumentation-mysql, for the MySQL drivers PyMySQL, mysqlclient and aiomysql.

The split between the core collection and the separate packages is that the core package only covers targets which can be exercised in-process, with no backend product or service needed to test against. Anything that needs a real server to test against, which the database and AWS packages do (their test suites run the real thing in a container), lives in a package of its own with its own release cadence. I also plan to have packages for Redis and MongoDB. Beyond that it will depend on what people are interested in seeing instrumentation for, with the only other thing I can see at the moment being the LangChain packages.

The posts so far

The posts are collected on the testing and tracing with wrapture guide page, which is the list that will keep growing, but for the record the reading order is as follows. First the starting point, on what wrapture is, why I built it, and how it was written:

• Introducing wrapture.

Then the unit testing side, where the difference from unittest.mock is that the real code still runs and everything that flows through it is recorded:

• Unit testing with wrapture.
• Recording calls with wrapture.
• Phased behaviour in wrapture.
• Beyond callables in wrapture.

And the tracing side, where the same bindings observe a running application instead, from a live call tree in a terminal through to spans in an OpenTelemetry backend:

• Live tracing with wrapture.
• Zero-code tracing with wrapture.
• Tracing Flask with wrapture.
• Finding slow code with wrapture.
• OpenTelemetry export in wrapture.

Learning it by doing

Reading about a library only gets you so far, so alongside the posts there is now a wrapture-workshops repository on GitHub containing 24 workshops you can work through to learn wrapture in an interactive workshop format. Each takes one thing you might want to do with wrapture and walks you through doing it in a live JupyterLab session, with the instructions in a side panel whose actions drive the session and check your work as you go. The early workshops track the blog posts, one per post, and the later ones go into areas the posts have not covered yet, such as using wrapture with pytest properly, converting an existing mock based test suite, async code, patching third party libraries, distributed tracing across two processes, and writing an instrumentation package of your own.

You need nothing installed to try them. The workshops can be run in a hosted environment on mybinder.org, a free public service that builds the repository into a temporary JupyterLab running in your browser. Building takes a minute or two. A session is discarded when it ends, so finish a workshop in the session you started it in. If you would rather run them locally, the repository README has the steps for doing that under any JupyterLab.

Each workshop installs wrapture into a virtual environment of its own inside the workshop directory, the way a project would, so nothing is left behind in the JupyterLab environment. The workshops are pinned to a released version of wrapture, so the documentation may at times describe something newer than what a workshop uses.

Beaten to the punch

My intention was always to write this summary post once the testing and tracing posts were done, but Simon Willison got there first with Don't sleep on wrapture, which was his second post on wrapture after covering the initial announcement back in August. I am not complaining. His reach on social media is a great deal larger than mine, so a lot more people will have seen his summary than would have seen this one, and that has been reflected in the way the number of stars on the GitHub repository jumped after his post went out. The Python Bytes team also featured wrapture on episode 494 of their podcast, which helped as well. Thanks to all of them.

Either way, this post still serves a purpose, since it is the one place with all the links, the package list and the workshops together, and it will be the post I point people at when they ask where to start.

A side note on the workshops

I do hope people try the workshops on mybinder, and not only for what they teach about wrapture. I have worked on Educates for many years as a way of hosting online interactive workshops, and it remains the platform I would reach for when a workshop needs a full Kubernetes-backed environment. Being able to deliver workshops inside JupyterLab, with instructions in a side panel that drive the session and check what you have done, is something completely new which I only wrote in the past week, as a JupyterLab extension called jupyterlab-workshop. A workshop is nothing more than a directory with a manifest and some Markdown pages, and it runs wherever JupyterLab runs, so mybinder can host it with no container or cluster of my own behind it. I will do some followup posts on that in the coming week or so, since it deserves more than a paragraph at the end of a post about something else.

12 Sep 2026 7:39am GMT

The pyopengl-video library is a small hack that lets you pass FrameBufferObjects (FBOs) directly to the local platform video encoding library (or allows you to use your existing video buffer as an FBO, depending on the platform). The purpose of the library is to allow an off-screen renderer to generate h264 in mp4 video previews of that off-screen rendering. I find that useful to allow an agent to capture e.g. gameplay demos or the like so that you can see what it is working on while you are not at your desk.

On Linux with nVidia uses NVENC. For linux with Intel or AMD uses vaapi. Windows uses D3D11 and COM. Requires PyOpenGL 4.0.0a5 or above. So far no OS-X solution.

12 Sep 2026 1:05am GMT

Everything in the last four posts rendered a trace for a person to read or wrote it to a file for later. The other destination is a tracing backend, fed while the application runs, and OpenTelemetry is the one that the ecosystem has converged on. wrapture treats it as a first-class destination rather than something you bolt on: the wrapture.otel subpackage ships in every wheel, and the otel extra brings the SDK and the OTLP exporter with it.

$ pip install "wrapture[otel]"

A plain install pays nothing for this, since nothing in base wrapture imports the subpackage until a config asks for it.

One table

Export is switched on by a top-level [otel] table in the same config file the Flask shop has been using. The presence of the table opts in, a service name identifies the process, and each signal's tuning nests beneath it. I shortened the metrics export interval so the demonstration would not have to wait a minute for a data point.

[otel]
service_name = "webshop"

[otel.metrics]
export_interval = 2

[[instrument]]
name = "flask"
ignore_paths = ["/health"]

[[observe]]
target = "shop:OrderService"
name = "place"
redact = ["card"]

[[observe]]
target = "shop:Gateway"
name = "charge"
redact = ["card"]

[[observe]]
target = "shop:Ledger"
name = "record"

Where the spans go is decided by the standard OpenTelemetry environment variables, so with a collector listening on the usual port nothing more is needed. For a look without a collector, the console exporters print the spans and metrics to standard output instead, which is what I used here:

$ OTEL_TRACES_EXPORTER=console OTEL_METRICS_EXPORTER=console \
    python -m wrapture -m flask --app webshop run --port 5003

One event becomes one span. A request becomes a SERVER span, a call or a block becomes an INTERNAL span beneath it, and the tree the printer drew is the tree the backend receives. For the quote of an item that is not in the catalog, the view's span arrives with error status and the exception recorded on it (trimmed here to the parts that matter; the real output includes the stack trace and the resource attributes):

{
    "name": "quote",
    "context": {
        "trace_id": "0x02f1fd262a7817d4f8b42fb0f6a30db3",
        "span_id": "0x5b94b62db087bbc9"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0xfba0a55ab3a322f0",
    "status": {
        "status_code": "ERROR",
        "description": "KeyError"
    },
    "attributes": {
        "wrapture.path": "webshop:quote",
        "wrapture.kind": "call",
        "wrapture.arg.item": "missing"
    },
    "events": [
        {
            "name": "exception",
            "attributes": {
                "exception.type": "KeyError",
                "exception.message": "'missing'",
                "exception.escaped": "True"
            }
        }
    ]
}

And the request span it is parented under:

{
    "name": "GET /quote/<item>",
    "context": {
        "trace_id": "0x02f1fd262a7817d4f8b42fb0f6a30db3",
        "span_id": "0xfba0a55ab3a322f0"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "status": {
        "status_code": "ERROR"
    },
    "attributes": {
        "http.request.method": "GET",
        "url.path": "/quote/missing",
        "http.route": "/quote/<item>",
        "http.response.status_code": 500,
        "wrapture.data.endpoint": "quote",
        "wrapture.data.remote": "127.0.0.1"
    },
    "events": [
        {
            "name": "exception",
            "attributes": {
                "exception.type": "KeyError",
                "exception.message": "'missing'",
                "exception.escaped": "False"
            }
        }
    ]
}

A few things in there are worth pointing at. The request span is named GET /quote/<item>, by the route pattern rather than the URL, because the Flask instrumentation annotates the request with its matched route once routing has run, and the exporter reads that as the semantic-convention http.route. A backend then groups by endpoint rather than seeing every URL as a distinct operation. The captured arguments and anything added with annotate() become span attributes under wrapture.arg.* and wrapture.data.*, with the card number already redacted before it got anywhere near the exporter. And the KeyError appears on both spans, once as the exception that escaped the view and once as the one noted against the request after Flask caught it, so the request span shows the 500, the error status and the reason together rather than a status with no explanation.

One trace across two processes

A trace within one process is only half of what a tracing backend is for. The trace-propagation example in the wrapture repository is two processes: a client that places orders against a quote service over HTTP, and the service itself, both observed by wrapture and both writing JSON Lines files. Every tree wrapture records carries a W3C trace id, minted at its root, and on the client side an instrumentation for urllib puts that id into the traceparent header of each outbound request. On the server side the WSGI middleware parses the header at the boundary, so that process's trees join the client's trace instead of minting their own.

The join needs no backend at all. Printing the first eight characters of the trace id from every line of both files, with the file it came from:

0b2ad016 server.jsonl backend:app
8f19b8c6 client.jsonl frontend:fetch_quote
8f19b8c6 client.jsonl frontend:fetch_quote
8f19b8c6 client.jsonl frontend:fetch_quote
8f19b8c6 client.jsonl frontend:place_order
8f19b8c6 client.jsonl urllib.request:OpenerDirector.open
8f19b8c6 server.jsonl backend:app
8f19b8c6 server.jsonl backend:quote
b1a416b0 client.jsonl frontend:fetch_quote
b1a416b0 client.jsonl frontend:fetch_quote
b1a416b0 client.jsonl frontend:place_order
b1a416b0 client.jsonl urllib.request:OpenerDirector.open
b1a416b0 server.jsonl backend:app
b1a416b0 server.jsonl backend:quote
...

Each order is one id across both files, client half and server half of one distributed trace. The repeated fetch_quote lines are the two blocks the client marks inside that function, which record under its path, and the lone server-only line at the top is a request that arrived with no traceparent header, which minted an id of its own at the boundary. The whole public surface the client instrumentation needed for this was wrapture.trace_headers(), which returns the pairs an outbound message made right now should carry, and is empty when nothing is being recorded, so injecting it is always safe.

Switching on [otel] in both processes changes nothing about the ids. The exporter claims the identity wrapture minted rather than minting one of its own, so the JSON Lines files, the outbound headers and the exported spans all read the same trace id, and the server's request span is created with the arrived identity as a remote parent. In the console output from the same run, the server's GET /quote/widget span carries the client's trace id and names the client's urllib.open span as its parent:

{
    "name": "GET /quote/widget",
    "context": {
        "trace_id": "0xcdde803e61b96f52e2eb3820c7004df0",
        "span_id": "0x9bb64b3bad9a6848"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": "0x670e42eb0ac7690a",
    ...
}

{
    "name": "urllib.open",
    "context": {
        "trace_id": "0xcdde803e61b96f52e2eb3820c7004df0",
        "span_id": "0x670e42eb0ac7690a"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x0e06d5674120b1db",
    ...
}

In a viewer, each order is one distributed trace with the service's request span attached beneath the outbound call that made it. One invariant governs the header handling and it is worth stating because it is the thing that makes this safe to switch on in a service that sits between other people's systems: never break a trace you do not understand. A header wrapture parses but nothing claims is forwarded verbatim, so an upstream product sees this service as a transparent hop, and headers wrapture does not parse are never touched at all.

Metrics for free

The traces signal exports events individually. The metrics signal aggregates the same events instead, and both were on above since the default is all signals. Request durations go into the semantic-convention http.server.request.duration histogram, attributed by method, route and status code, and observed calls go into a per-path wrapture.call.duration histogram whose error series split out by exception type. From the same run, the attribute sets on the request histogram's data points were:

{
    "http.request.method": "POST",
    "http.route": "/order",
    "http.response.status_code": 200
}

{
    "http.request.method": "GET",
    "http.route": "/quote/<item>",
    "http.response.status_code": 500,
    "error.type": "KeyError"
}

So per-endpoint latency and error rate read straight off the histogram with no code involved. The reason the bound path is safe as a metric attribute where a raw URL would not be is that the config chose the bindings, so the set of values is closed; requests are attributed by route pattern for the same reason, never by URL. The design is the Aggregate collector from the previous post with the aggregation handed to the SDK: bounded memory, no values captured, nothing retained.

What it costs

The usual objection to instrumenting Python is the overhead, so this is worth a paragraph. Exporting through wrapture costs about the same as instrumenting with the OpenTelemetry SDK directly, and on a call that raises it costs noticeably less, because the SDK's record_exception formats the stack trace through traceback.format_exception, which on current Pythons parses each frame's source to draw caret underlines that no backend renders, and wrapture's sink formats the same frames without them. The design point behind the rest is that the sink does not use the SDK's tracer at all. Everything a span needs is known when its event closes, so the sink builds the finished span at that moment and hands it to the SDK's own processor, skipping the tracer's mutable span object with its validated attribute store and locks, which is where most of the per-span cost used to go. The measured figures, with the methodology, are in the cost section of the OpenTelemetry export page, and I would rather point there than quote numbers that will be out of date by the time anyone reads this.

The thread from the beginning

When I introduced wrapture I said there were two interests behind it, correctness in testing and instrumenting programs for tracing, and that underneath they wanted the same thing: a way to see the real calls as they happen. This is where the second one ends up. The three bindings on the shop have not changed since the first testing post. In a test they feed a tape and the assertions read off it. In production they feed a backend, with a request as one tree, a trace id that survives crossing to another service, and metrics aggregated from the same events. The only thing that changed along the way is who was listening.

The OpenTelemetry export page has the rest of the table, including sampling, the logs signal and how wrapture's pipelines coexist with an application that already uses the OpenTelemetry API on its own account.

12 Sep 2026 12:00am GMT

---

News

PyCharm & Django Fundraiser Extended to September 14

The second half of our annual JetBrains fundraiser has been extended through September 14, 2026. Thank you to JetBrains for the extra time. You still have time to renew your PyCharm license or give it a try.

Call for volunteers: Fundraising Working Group

The Django Software Foundation is looking for people to join the Fundraising Working Group. This is a particularly interesting time to get involved.

---

Django Software Foundation

Django Steering Council Meetings - 2026

Notes from the September 7 meeting: packaging related tools as extras, experimental features, usage telemetry, and Fellows pinging the CompositeField and content type parsing DEPs.

---

Python Software Foundation

Incident Report: File Hosting Errors

Two weeks of intermittent 502s and 503s on files.pythonhosted.org came from a Fastly canary that left one cache node half rolled back, which then exposed three latent bugs in PyPI's own config. The takeaway for the rest of us: turn on dependency caching in CI, which setup-python leaves off by default.

---

Wagtail CMS News

Prototyping a new CLI for Wagtail

Thibaud Colas is prototyping wagtail-cli, a terminal interface over the v3 API for browsing and publishing pages, managing media, and scaffolding projects, partly so AI agents can reach the CMS without driving a browser. Try it with uv tool install wagtail-cli.

---

Updates to Django

Today, "Updates to Django" is presented by Raffaella from Djangonaut Space! 🚀

Last week we had 15 pull requests merged into Django by 12 different contributors - including 2 first-time contributors! Congratulations to Vimal Sahani and Dave Gaeddert for having their first commits merged into Django - welcome on board!

News in Django 6.1:

• Fixed a regression that caused __in lookups on annotations to erroneously return empty querysets and __range lookups to crash when passed an iterator (#37311).
• Fixed a regression where custom querysets used with Prefetch for forward foreign key or reverse one-to-one relationships were not routed using the parent queryset's database (#37300).
• Fixed a regression where an admin changelist search crashed when a search_fields entry used an __exact lookup on a field with choices, and where any search term matched all rows with a True value when an __exact lookup was used on a BooleanField (#37263).

News in Django 6.2:

• Omitting the tzinfo argument of Extract and Trunc database functions in migrations when USE_TZ is True is deprecated. Pass tzinfo explicitly to avoid inconsistent behavior if TIME_ZONE changes.
• Unsaved instances with a composite primary key or a db_default primary key no longer compare equal to other instances.

---

Django Fellow Reports

Django Fellow Report - Jacob

An early report for about a day at my desk before heading out for some vacation. Six tickets reviewed, two authored, and the usual misc.

Django Fellow Report - Sarah

Seven tickets reviewed and another seven authored. Fellows sync, engaging with security issues, and website working group meeting.

Django Fellow Report - Natalia

Post-DjangoCon US week (including the emotional low 💔 that comes with it), with most of my time going into two things: first, iterating on a security report until we could confirm the issue, followed by developing a solution for it. I also continued the calendar versioning work around DEP 20 📅, including both the Django implementation and the release process updates. And, after iterating on a PR tutorial since Vigo, I finally got to see it through.

---

Sponsored

Your task ran before the transaction committed.

django-ox is a production worker for Django Tasks that runs on the database you already have: no Celery, no Redis, no broker to babysit. Enqueue is a plain INSERT, so a task commits or rolls back with the data it belongs to. Django 5.2 LTS and 6.x, free, BSD-3.

---

Articles

Soft-deprecating re.match()

After 30 years of tripping people up by anchoring at the start of a string but not the end, re.match() is soft-deprecated in Python 3.15 in favor of the clearer re.prefixmatch().

1001 Django apps - The myth of a well-structured Django project

Ronny Vedrilla makes a thoughtful case for fewer Django apps: treating them as database namespaces rather than folders, starting with one domain plus a few "satellite" apps, and using import-linter to enforce the boundaries so they don't just become decoration.

Nifty Django Feature: Q() Objects

Assigning Q() objects to named variables makes filter logic readable and reusable, and passing several into one .filter() call avoids the extra joins you get from chaining.

---

DjangoCon US Recaps

I Think That Feeling is Called Hope - Rachell Calhoun

Rachell Calhoun's DjangoCon US recap comes from inside the machinery: a third year chairing volunteers alongside Monica Oyugi, the first time all five Djangonaut Space founders stood in the same room, and open spaces on contributing to Django and on music. It ends on the case for volunteering, which she calls the fastest way she knows to stop feeling like a stranger at a conference.

TSBT73: Pumpkin Spice Bytes - Velda Kiara

Velda Kiara's update on DjangoCon US and related tech discoveries.

The Community Behind Django: My First DjangoCon US Recap and Highlights - Seyram Theresa

Theresa's very in-depth recap of DjangoCon US, from talks and keynotes to lightning talks, hallway convos, Chicago adventures, and more.

DjangoCon US 2026 | Chicago - Jon Gould

Recruiter Jon Gould's fourth DjangoCon, from the sponsor side of the table. His favorite parts were the unscheduled ones, like Aman Singh's early morning walks to the Bean.

Your City Is a Spatial Database and Nobody Told You - Jason Judkins

Jason Judkins's recap of a specific talk at DjangoCon US the other week, by Drishti Jain, highlighting how important shapes and GeoDjango are in the real world.

---

Django Job Board

Two construction-AI roles at Provision and a backend seat on the platform that runs a family-owned cruise agency.

🆕 Machine Learning Engineer (Hybrid) at Provision

Django Developer at The Cruise Brothers

Full Stack Software Engineer (Hybrid) at Provision

---

Projects

gettranslatebot/translatebot-django

Translates .po files and model fields with an LLM, but only the new and changed strings, using a TRANSLATING.md glossary in your repo to keep terminology consistent between runs. Placeholders and HTML tags survive intact.

lincolnloop/django-absurd

Plugs Absurd, a Postgres-native workflow engine, into Django's Tasks framework so background tasks and durable workflows run on the database connection you already have. Needs Django 6.0+ and psycopg 3.

11 Sep 2026 3:00pm GMT

Weeknotes (2026 week 37)

Tonight the temperature will drop to 10°C. I like it when it's hot, but now I really enjoy the slightly cooler temperatures in Europe. The last post was written in the middle of the heat wave. Yesterday we still had temperatures of over 30°C, which is a lot for September. But it will be the new normal. Next year probably won't be cooler.

Releases from the last four weeks

Lots of activity!

django-authlib

django-authlib 0.19 hardens the OAuth2 implementation a bit against replay attacks, provides utilities for removing password logins from the Django administration interface, and removes some of the confusion around role-based permissions. Upgrading is recommended.

django-content-editor

django-content-editor 9.0.2 fixes a bug in the function which allows cloning plugins from other regions. The bug was most visible when using something like django-json-schema-editor's plugins which are only proxy models. Their type was lost because I was using _base_manager, which didn't do the downcasting properly. (Sorry for the word salad.)

django-tree-queries

django-tree-queries 0.26.1 now uses annotations instead of .extra() to add the tree_path and tree_depth fields. This is great because the ORM knows these fields properly and we can now use e.g. .filter(tree_depth=1) instead of .extra(where=["..."]).

This change was mainly motivated by the renewed interest in deprecating .extra() at some unknown future point in time.

django-debug-toolbar

django-debug-toolbar 8.0 ships a new design! I again didn't contribute a lot, but I'm very proud that we were able to ship this as a team.

feincms3

feincms3 6.0.2 ships new utilities and, most importantly, fixes crashes when passing %00 to the root middleware. Using null bytes when comparing text field values on PostgreSQL leads to crashes. These are mostly an annoyance without data exposure, but definitely worth fixing.

The reason for the major version bump is that we finally dropped support for Django 3.2. That's the only breaking change, so upgrading should be easy for everyone.

django-js-asset

django-js-asset 4.1 fixes issues around lazy CSP nonce handling as well as the same issue Django itself had around rendering HTML-safe strings.

django-prose-editor

django-prose-editor 0.27.2 adds a default menu item for the code block extension and updates ProseMirror and Tiptap. It also adapts to the changed CSS of the Django 6.1 administration interface.

django-json-schema-editor

django-json-schema-editor 0.14.1 includes a fix for saving JSON schema fields when the schema doesn't contain any properties, plus fixes for the Django 6.1 admin CSS.

django-admin-ordering

django-admin-ordering 0.21 now activates orderables properly when the same model was orderable both in the change list and in the change form. I developed the fix for this in early 2025 but never released it. It's what happens.

feincms3-cookiecontrol

feincms3-cookiecontrol 1.7.2 includes bugfixes for edge cases while bringing down the byte count for the cookie banner and media embedding script from 3961 (in 1.7.1) to 3909 bytes.

feincms3-data

feincms3-data 0.11.1 brings fixes for handling unique fields when the same unique value is removed and then re-added with a different primary key.

django-translated-fields

django-translated-fields 0.14 contains no relevant code changes to the module itself at all. It was just time to release a new version more than two years after the last release to show that the project is still active and update the Trove classifiers.

09 Sep 2026 5:00pm GMT

Over the summer I published a series on coding tactics: the everyday craft of ifs, loops, and the reasoning behind them. Eight posts, one thesis, best read in order. This is the map.

08 Sep 2026 10:00am GMT

I get asked this rhetorical question a lot, in various forms:

Sure, datacenters might use a lot of energy, but you don't have to use a hosted frontier model to do software development. What if I just run a local open-weights model to do some coding, with an open-source coding agent? Video games also use my GPU. Is local model development any worse than playing a video game?

So I want to write down my comprehensive answer to this: Yes, using an LLM to write some code is worse than playing a video game, for a few reasons.

Video Games Are Interactive, LLMs Are Batch Jobs

Video games use compute to respond to human input. You are using your GPU while you are looking at a screen, displaying an image. When you are done playing, you shut off the game, and your computer goes back to idle. It's much less energy. By contrast, agentic loops with evals (the only kind of "AI" that is meaningfully any good at coding) are running hot, for days. To use the most recent example of such a thing, a very rough first sketch of an implementation of a Windows graphics API backend to help port a paint program to other platforms, it took 3 weeks of Claude time, "day and night". Do you play a lot of video games for 500 hours to make it past the tutorial level, while also using other computers for other things, as well as the rest of your carbon footprint?

Video Games Need Development, LLMs Need Training

Video games use compute to respond to human input during development, too. Your game has to be made, but your LLM has to be trained. LLMs use a historically extreme amount of power, probably using more than the entire Internet, but it's kind of hard to say. Still, it seems a reasonable estimate to within several orders of magnitude that even over a multi-year project with hundreds of developers, the power used to develop an individual video game is nowhere close to training even a small LLM.

This is true even for local models. OpenAI has openly claimed that DeepSeek "stole its intellectual property", and I have heard grumblings that none of the open-weights generalist models could realistically exist without the massive lift that the frontier labs are doing with their training, in various other ways too. Secrecy throughout the industry makes this kind of impossible to understand rigorously, but it seems fair to say that you are partially culpable for all that famously energy-intensive frontier lab training if you're using a local model.

And They Keep Needing Training

You also can't dismiss this as a sunk cost, because in order to stay current with industry developments, models need to be updated with new information from the rest of the world, which means that you need to keep training them. Beyond the energy for your own use, if you want a real-life agentic workflow that actually does useful stuff, practically speaking you would still need to update your local models over and over again, at least once every few months, which means you would be incentivizing continued energy consumption by whoever was doing that training for you, including the energy cost of scraping.

Let's Be Real Here, You Aren't Actually Using A Local Model

This question is a hypothetical thought experiment. Despite synthetic benchmarks that keep showing there isn't much difference between open weight and frontier models, nobody's actually using local models for much of anything beyond sharing those talking points. Depending on which benchmark you're looking at, maybe it's good enough or maybe it's worse.

As an inveterate AI hater, all these systems seem pretty bad to me, but it seems that people who find them useful tend to subjectively believe the frontier models are worth the premium, and that's what they're actually using. Once you have accepted that it is OK to use LLMs for coding at all, it seems like a very quick slippery slope on down to "we'll go ahead and use the frontier models for now anyway, but we could be ethically better in the future by switching to an open weights one, that option is always available".

There's A Reason We Have Data Centers

Devolving power usage to local LLMs might be good to make users responsible for their costs and decrease the impacts to communities that are physically next to huge concentrations of power utilization, not to mention generation. However, there's a reason that it makes sense for the providers to build these giant facilities: economies of scale reduce total power consumption, they don't increase it. If you do all the same stuff with a local model that they have to do in hosted environments, it will probably take more power, even though you will be incentivized to do different stuff. This incentive to "do different stuff" is why although local models can hypothetically hold their own against the frontier labs for some tasks, when people or businesses take their inference costs in-house they often find that it's too painful and move back to hosted LLMs.

There Are Problems Other Than Power

These are subjects for a different post, but you have to consider a lot of other externalities: AI psychosis, de-skilling, comprehension debt, cultivating a dependency, introducing security defects, limiting your design space based on what LLMs can understand, context rot, wasting time on invalid solutions, introducing unpredictability into your workflows. You still have to consider the total cost benefit ratio.

To Sum Up

Local LLMs might alleviate some of the harms from using the hosted frontier providers. There are fewer privacy concerns, you can measure your power utilization and be more directly responsible for it, you can build interfaces with affordances that are less oriented towards addiction and dependency than the major frontier labs' harnesses.

But they're not automatically "the same as playing a video game" just because they can use the same GPU.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

06 Sep 2026 10:57pm GMT

Starting with 0.3.0, Astral's uv brought many great features, including support for cross-platform lock files uv.lock. Together with subsequent fixes, it has become Python's finest workflow tool for my (non-scientific) use cases. Here's how I build production-ready containers, as fast as possible.

06 Aug 2026 12:00am GMT

As I have discussed in previous posts, "AIs" can make mistakes. In fact, they do make mistakes, and their mistake-making patterns are such that where and how they will make mistakes is both uncertain and constantly changing.

Thus, in any scenario where you want to attempt to make "productive" use of "AI", you must have a system in place for checking every result. Not checking some results; checking every result. If each result might have a consequence for you (and if it didn't have a consequence, why bother automating it?) and you cannot predict in advance which kinds of results will need verification, then verification is always required.

The verification often ends up being just as expensive as doing the work in the first place, which means that if you want your usage of "AI" to be personally profitable, you have to find someone else to externalize the cost of verification onto. This person becomes your adversary, and, if you are successful, your "AI's" victim.

The Ladder-Climber And Their Reverse-Centaur Rungs

One way that this constellation of facts can straightforwardly assemble themselves into a dystopian nightmare is the phenomenon, described by Cory Doctorow, of the reverse centaur. This is when your employer non-consensually turns you into the verification system. The "AI" does the fun part of initially performing the work, and then you do the boring part where you check if the robot is right and clean up its messes, even if everyone already knows that it would, in aggregate, be cheaper for you to do the work in the first place.

Reverse centaurs can be made from any automation, not only "AI" automation. I think that there is a reason that this term happens to have emerged in the "age of AI", though, and not with earlier automation technologies (even those which were considerably more viscerally horrific). That reason is: the wrongness of "AI" output is not merely a technical feature that must be compensated for, it is a generalized externality.

As I mentioned above, if you are responsible for the entirety of the work, both extruding the "AI" output and checking it, it's usually cheaper to have humans do the entirety of the work to begin with. When humans do the writing directly, we can check as we go, and thus verification doesn't need to be as comprehensive.

When "AI" coding advocates say "code review is the bottleneck", what they are observing is that the LLM is still rolling the dice for each PR, and a human is still necessary to verify that each of those rolls is a winner. But calling this process "code review" is a bit of a misnomer; it's not really "code review" in the traditional sense, it's human understanding.

Before the advent of "AI", the human understanding was implicit in the process of writing the code in the first place¹, and the code review was a way of diffusing and extending that understanding. Now that the code can be authored with no initial understanding taking place, that cost has not gone away, it has moved.

Human understanding was always the bottleneck.

However, this is taking a collaborative view of a software project, where satisfying the needs and solving the problems of your customers are the goals. We can see that "AI" is a bad tool to satisfy those goals, because all it's doing is converting the first half of the work, that of understanding the code as you write it, to understanding the agent's output as you read it.

What if, instead, we were to take the view that every software company is a Hobbesian nightmare, red in tooth and claw? In this view, the only goal of a software project is for the individual developers to make their promo cycles and get their bonuses. Given that there is only a certain amount of money to go around, this is a zero-sum game where each programmer wants to look more productive than their colleagues.

Pretty much every organization finds it easy to reward "productivity" as expressed by lines of code emitted, but the benefits of doing thorough and thoughtful design, analysis, and code review very difficult to reward. In this world, an LLM is an invaluable tool for the sociopathic ladder-climber, particularly if your legacy organization is still structuring their workflows as if the person prompting the bot is "writing" the code, and then they get to foist off the act of "reviewing" the code onto someone else.

Here, the prompter effectively externalizes the cost of the LLM's failures but internalizes any benefits. The prompter will vibe-code a big feature, so large that the assigned reviewer can't possibly comprehend it all effectively. When this happens, the reviewer will, eventually, be pressured to approve it, even if they can try to spot a few problems along the way. The reviewer has their own work to get back to, after all, the obligation to review the prompter's (read: the bot's) code is a drain on their time that they are not going to get rewarded for.

If this feature is a big success, the prompter gets a promotion. If it causes a big issue, well, the reviewer must not have been careful enough.

This is why LLMs are "good for coding", and also why their biggest promoters keep having outages.

The Generative Gish Galloper

Coding is the biggest "success story" of this type of adversarial communication, but it is by far not the only instance of such a thing. LLMs create a new form of leverage that can turn Brandolini's law from a linear advantage into an exponential one. If you are engaged in a political debate where you want to overwhelm the other side in nonsense, an LLM can generate bullshit faster than it is physically possible for a human being to type, let alone respond thoughtfully. There is an asymmetry to the utility of this weapon as well: only one side of the political spectrum wants to flood the zone and destroy trust in institutions and the concept of truth. There's a good reason that the fascists love it.

Straightforward Spam and Fraud

This is kind of obvious, but LLMs can generate lightly-customized, plausible-looking text much more quickly than any human being. This facilitates their use in fraud, spam, and scams. In a spamming or fraudulent interaction, once again, the costs are externalized onto the victim: the recipient of a spam message has to do all the work of "checking" the LLM's output. Spammers already expect very low hit rates from boilerplate, and if the LLM can increase those percentages from 1% to 5% the technology will pay for itself; they don't need anything like reliable accuracy.

Customer "Support"

If you have any kind of commercial relationship with a company, I probably don't even need to mention this: customer "support" bots are a misery. Everybody knows it at this point. But customer support is usually conceptualized by businesses as an adversarial interaction, because it is a cost center. They maintain internal metrics on time-to-resolution and try to optimize them. Implicitly, this creates a dynamic where the goal of the customer service agent's job is not to solve your problem, but to emit noise that will cause you to think your problem is resolved, or to give up, as fast as possible. Unsurprisingly, LLMs can emit this noise faster than humans can, getting those customers off the phone. But those customers will remember those interactions, and the story outside the TTR metrics is horrible.

Similarly to the situation in software development, LLMs can look very good on paper for customer support, but mostly what they are doing is illuminating the problems with the industry's existing metrics, by turning "winning the metrics battle against the customer" into a more obvious and immediate defeat for the company's long term reputation.

"Education"

In 2026 it is sadly a fact of life that students cheat all the time using "AI", and that this cheating is very successful, in that the teachers find it very hard to detect.

LLMs are great for cheating on schoolwork because the student is externalizing the work of the checking onto the teachers, who are often starting at a disadvantage to begin with, at least in the US.

My view is that this is happening because of a divergence in the way that students vs. teachers (or, more accurately, "the broader educational system") view grading.

When a student is asked to write an essay, the teachers see the effort as both intrinsically worthwhile for the student, as well as useful as a pedagogical tool to evaluate and react to the student's progress. The student, by contrast, sees a stumbling block designed to knock them off the path to success and into a permanent underclass. It is no wonder that the student sees "AI" as useful to their own goals and has no compunction about deploying it.

There is a bitter irony that the ability to understand the inherent value of actually writing the essay on their own is the sort of thing that students can really only learn by writing a bunch of essays. There's no way that I can think of which makes the benefit legible as long as a shortcut is available.

The net effect here is a downward spiral, where the already-wobbling educational system is sustaining an attack that it doesn't have the resources to recover from. The individual students' attacks against their teachers and their schools' grading systems might appear to momentarily succeed, but they will win the battle and lose the war.

Spamming "For Good"?

Usually when we talk about someone unilaterally choosing to enter into an adversarial relationship, that's an "attack" and for good reasons we have a negative impression of the attacker. However, I would be remiss if I did not point out that there are some cases where the relationship was already adversarial; just because you're the attacker doesn't mean that you are evil.

For example we might imagine use-cases like automatically filing appeals for prior authorizations against health insurance. It's relatively well-known at this point that the main way for-profit insurers maintain their margins is by denying claims right up to the line of the policies themselves being fraud, so using a spamming tool to fight them might be entirely justifiable² in that case.

Similarly, using an LLM could be justified in a fight against a company refusing to honor a warranty. One could imagine using an LLM to immediately generate replies and escalations.

However, even in imagined cases like these, the underlying problem is that the insurers and the vendors already have a tremendous amount of structural power, so it is more likely that they will have the advantage in deploying a communications weapon like an LLM, as well as enacting policies to simply ignore any LLM-based communication that you might submit. Worse, if these strategies were to become widespread, they might provide an excuse to reject any communications by feeding them into an unreliable "LLM detector" and issuing an automated "computer says no" even to hand-written correspondence.

It is also worth stressing that these cases are imagined, as compared to the very real coworker-abuse, spam, scam, fraud, and disinformation campaigns being waged in real life today.

Therefore, while legitimate uses might exist, it's hard to imagine that there's anywhere they would be genuinely valuable and sustainable. In the best case "AI" will provide a temporary advantage for underdogs that will provoke an arms race which the resource-advantaged adversaries will win in the long run, in the worst case the arms race itself will cement permanent structural change that will make things worse.

"Search" By Stealing

Most of the adversarial utility of "AI" is on the "write" side, since write-amplification is more obviously aggressive than reading. But the "read" side of LLMs - summarization and question-answering - can be a form of attack as well.

To begin with, the act of reading itself is currently enormously destructive, but that's arguably not a fundamental aspect of this technology. They could set reasonable rate-limits and respect things like robots.txt, as search engines have for decades now. They could also refrain from committing criminal levels of copyright infringement. But, today, using "AI" tools does suborn this sort of out-of-control crawling.

More insidiously, consider the scenario described in this YouTube video. The LTT Bros decided to try Linux again, and in the course of so doing, they had problems. When trying to solve these problems, they were faced with a choice: they could consult Reddit, or they could ask an LLM. Asking an LLM would "gaslight the heck out of" them, but they still found it preferable, because they would at least get an answer without getting yelled at.

Initially this sounds great. But it also means that you want to extract knowledge from a community, while mechanically eliding any values or norms that the community may want to impart as part of offering that knowledge. As someone who spent many years in a community tech support role, this is worrying. Many requests for support are people asking how to do things that will momentarily solve a superficial problem but create a long-term reliability problem or even an immediate security risk, that the question-asker doesn't want to hear about. Consider the question "I'm tired of entering my password so much, how do I make it so my laptop unlocks automatically". An obsequious chatbot will helpfully tell you how to do this without pushback.

But, this is also a sort of ethically murky area. The Linux community is somewhat famously, for many years now, a toxic cesspool of general hostility, misogyny, etc. It is certainly a good thing that people can get access to this knowledge without subjecting themselves to abuse. But it also means that the people with the power and the privilege to change the community for the better can just quietly withdraw, rather than fixing the problems. It also means that the positive elements of culture cannot be transmitted, and people will have no opportunity to learn about unknown unknowns.

In this case, the "adversarial" communication is with society. The thing that using an LLM for search lets you do is withdraw from society and avoid forming any personal connections. There are some personal connections which are painful and annoying, and so that can feel like a momentary balm. But the need to make connections in general is, like, the concept of society itself.

Who Am I Hurting?

LLMs are good at adversarial communication. They are so good at it, relative to their other benefits, that they will tend to make communications adversarial if you are not remaining vigilant about the possibility that it might do so. My request to you, dear reader, if you are going to use such tools, is to always ask yourself, "who might I be hurting, if I use an LLM for this?"

If you're using an "AI", who is its adversary? If you haven't given it one yet, who might the "AI" turn into an adversary? Who might you overwhelm with an asymmetric amount of output, or, if you're receiving information and not sending it, who are you taking that information from without consulting?

Figure out the answers to these questions and conduct yourself accordingly; the answer might be "yourself".

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

23 Jun 2026 8:06pm GMT