ZAR: A New Archive Format with a Python API

Have you ever been bored because of slow compression/decompression tools? Because speed matters, I've developed a new archive format with very fast tools.

ZAR is a new archive format, not just another Python wrapper around ZIP or TAR. It is designed to combine fast parallel Zstandard compression with features that the familiar formats don't offer together: content deduplication, versioned commits, historical restores, and recovery checks.

The format is implemented in Rust and comes with a native Python API. Create archives quickly, keep earlier file versions in the same archive, and restore part of the archive without unpacking everything.

And Python is just one way to use it: the same Rust library also powers a command-line tool, a desktop archive manager, and bindings for many other languages.

What makes the format different?

Most archive formats are great at packaging files. ZAR is built to manage changes to those files over time, too. Each update can add a commit while preserving earlier states, so you can inspect or restore history without keeping a separate full archive for every version.

These are not just checkboxes: commits let you update an archive without discarding the old state, deduplication avoids storing duplicate content within a create or add operation, and block framing lets ZAR compress in parallel and verify individual blocks. The result is a format for people who want both quick archive operations and useful history.

Why Python developers should take a look

🚀 Fast compression, without leaving Python

The Python package, pyzarlib, is a PyO3 extension for the Rust archive library. ZAR splits data into blocks and compresses them in parallel with Zstandard. Long-running Rust operations release Python's GIL, so other Python threads can keep working while an archive operation is running.

In one unencrypted, hot-cache benchmark on a Linux Core i7-10700, ZAR compressed a 525 MiB Linux source tree in 0.86 seconds. In the same run, ZIP took 11.01 seconds and 7z took 24.91 seconds. ZAR's archive was 115.5 MiB-smaller than ZIP's 151.1 MiB, though not as small as 7z's 81.1 MiB. If you need a much smaller archive, 7z may be the better fit; if you want a compelling balance of compression time and size, ZAR is worth a try.

One hot-cache run on a Linux Core i7-10700; results vary by machine, storage, and data.

The full benchmark report includes extraction times, cold-cache runs, and details about the test setup.

🐍 Python API - and a CLI for Python workflows

Create, update, inspect, verify, and extract archives directly from Python. Inputs accept pathlib.Path as well as strings:

from pathlib import Path

import pyzarlib

archive = Path("backup.zar")

created = pyzarlib.create_archive(
    archive,
    [Path("src"), Path("README.md")],
)
print(f"Created with {len(created['warnings'])} warning(s)")

# Add a new commit. Earlier file versions remain in the archive history.
pyzarlib.add_archive(archive, [Path("README.md")])

# Store and read custom archive-level metadata.
pyzarlib.update_custom_metadata(
    archive,
    set={"project": "example", "revision": 1},
)
metadata = pyzarlib.read_custom_metadata(archive)
print(metadata["project"], metadata["revision"])

for entry in pyzarlib.list_archive(archive):
    print(entry["path"], entry["type"])

check = pyzarlib.check_archive(archive)
print(f"Verified {check['commits']} commit(s)")

# Preview a restore before writing files.
plan = pyzarlib.extract_archive(
    archive,
    "restored",
    dry_run=True,
)
print(f"Would extract {len(plan['files'])} files")

Custom metadata values can be strings, signed integers, or finite floats. Updating metadata appends a metadata-only commit; keys beginning with zar. are reserved. Metadata updates require an unencrypted archive.

To append an encrypted commit to the archive above and read its file back:

from getpass import getpass

password = getpass("Commit password: ").encode("utf-8")
pyzarlib.add_archive(
    archive,
    [Path("secrets.txt")],
    password=password,
    encryption_scope="header-and-data",
)

contents = pyzarlib.extract_archive_to_memory(archive, password=password)
print(f"Read {len(contents['secrets.txt'])} bytes")

Only the new commit is encrypted; earlier commits remain unchanged. Without the password, commits with encrypted headers are skipped, so a listing or extraction can show only part of the archive. For commits with different passwords, pass a passwords mapping from commit IDs to password bytes.

Need a terminal workflow? The package also provides zarc-py, a Python CLI for creating, adding, removing, joining, undoing, checking, listing, inspecting, and extracting archives.

The package supports Python 3.11 and newer, with CI coverage for CPython 3.11-3.15 and free-threaded CPython 3.14t and 3.15t. Native work releases the GIL, and the extension is tested with concurrent Python calls.

🗜️ Compression controls and familiar archive inputs

Choose a Zstandard compression level from -7 to 22 and a block size from 64 KiB to 32 MiB, or let ZAR select a block size automatically. Default compression level is 3. Files with identical content can be deduplicated within a create or add operation. Some already-compressed formats and media are stored without an extra compression pass to improve compression speed.

You don't have to start with a ZAR archive. Python create_archive and add_archive can import entries from ZIP, 7z, RAR, ISO, TAR, TAR.GZ, TGZ, and GZ files, preserving their internal paths. Encrypted ZIP/7z/RAR inputs and some special archive entries are not supported; see the project documentation for the precise limits.

More than a compressed file

ZAR archives are built around commits:

• Add new or changed files without throwing away earlier versions.
• Arbitrary metadata as key-value pairs can be added to the archive and is saved in metadata only commits.
• Remove paths while retaining their previous contents in history.
• Undo trailing commits or join a commit range to compact history.
• List and inspect archive metadata without decompressing file contents.
• Extract one file or read file data into memory, instead of unpacking the whole archive.
• Filter files and folders by patterns during creation, listing, and extraction.
• Encrypt headers, data, or both with AES-256-GCM or ChaCha20-Poly1305.

That makes ZAR useful for more than a one-off download bundle: it can serve as a compact, inspectable history of project snapshots or application data.

One Rust core, many ways to use it

The shared archive library is available through:

You can use the zarc command-line tool, manage archives in the zarg desktop application, or connect an AI client to the local zarmcp server. The companion zcopy tool transfers file trees over a network using compressed, optionally encrypted ZAR batches. Try it out. You'll be impressed how fast you can transfer big file trees over the network without creating (temporary) archives.

Installation

Build the Python extension from the project source with Maturin:

python -m pip install maturin
cd pyzarlib
maturin develop --release

Then import pyzarlib in your Python code. The repository includes Python installation and API documentation, examples, and tests.

Get started

If you're a Python developer who wants quicker archives without giving up a comfortable Python API, try ZAR on a representative dataset. The benchmark includes machine and method details so you can compare it against your own workload-not just trust a headline number.

Explore the source, complete documentation, and benchmark reports:

https://github.com/brmmm3/zar-rs

Licensed under the MIT or Apache-2.0 License.

15 Nov 2026 7:00am GMT

The last post was about muructl serve as the main process of a container: running the workers it is told to fit the container, answering probes on a port of its own, shutting down cleanly on SIGTERM, and running with a read-only root under any user ID. None of that cares what runs the container. Most of these containers end up on Kubernetes though, and Kubernetes has its own way of doing several things a Python web server would otherwise need help with. Rolling out new versions, spreading traffic, scaling, running one-off tasks, all of that is done by the platform already.

So the aim with Muru on Kubernetes is to fit in with what Kubernetes provides rather than duplicate it, and to fill the gaps where what Kubernetes provides doesn't quite reach the application.

The Deployment

The image is built the same way as in the last post, with a small WSGI application which has a page that waits as if on a database. The Deployment follows the one in the Muru documentation:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop
spec:
  replicas: 2
  selector:
    matchLabels: {app: shop}
  template:
    metadata:
      labels: {app: shop}
    spec:
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsNonRoot: true
        seccompProfile: {type: RuntimeDefault}
      containers:
        - name: shop
          image: muru-blog-k8s:1
          imagePullPolicy: IfNotPresent
          env:
            - {name: MURU_SERVE_PROCESSES, value: "2"}
            - {name: MURU_SERVE_THREADS, value: "4"}
            - {name: MURU_SERVE_SHUTDOWN_DELAY, value: "5s"}
            - {name: MURU_SERVE_SHUTDOWN_BUDGET, value: "25s"}
            - {name: MURU_SERVE_LOG_FORMAT, value: "json"}
            - {name: SHOP_DATABASE, value: "postgresql://db.shop/shop"}
            - name: POD_NAME
              valueFrom: {fieldRef: {fieldPath: metadata.name}}
            - name: POD_NAMESPACE
              valueFrom: {fieldRef: {fieldPath: metadata.namespace}}
            - name: NODE_NAME
              valueFrom: {fieldRef: {fieldPath: spec.nodeName}}
          ports:
            - {name: http, containerPort: 8080}
            - {name: admin, containerPort: 9090}
          startupProbe:
            httpGet: {path: /startupz, port: admin}
            periodSeconds: 2
            failureThreshold: 60
          livenessProbe:
            httpGet: {path: /livez, port: admin}
          readinessProbe:
            httpGet: {path: /readyz, port: admin}
            periodSeconds: 2
            failureThreshold: 1
          resources:
            requests: {cpu: 500m, memory: 512Mi}
            limits: {cpu: "2", memory: 1Gi}
          resizePolicy:
            - {resourceName: cpu, restartPolicy: NotRequired}
            - {resourceName: memory, restartPolicy: NotRequired}
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities: {drop: ["ALL"]}
          volumeMounts:
            - {name: run, mountPath: /run/muru}
            - {name: tmp, mountPath: /tmp}
      volumes:
        - name: run
          emptyDir: {sizeLimit: 2Gi}
        - name: tmp
          emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: shop
spec:
  selector: {app: shop}
  ports:
    - {name: http, port: 80, targetPort: http}

Much of this is what the last post covered, now expressed as Kubernetes configuration. All the settings come from environment variables. The probes go to the admin port, which the Service doesn't expose. The pod, namespace and node names come in through the Downward API so they end up in every JSON log record. I labelled the namespace to enforce the restricted Pod Security Standard and the pods were admitted without any warnings, as Muru only needs /run/muru and /tmp to be writable.

As in the last post, the number of processes and the threads in each are set explicitly, chosen to go with the pod's resources. With a CPU limit of 2 and a memory limit of 1Gi, that is two workers of four threads:

container limits: 2 CPUs from cpu.max 200000 100000, memory.max 1.0GB with 128MB of headroom
...
serving app:application on http://0.0.0.0:8080 with 2 workers of 4 threads

If the limits change, so should these. The memory limit is also what the memory guard from the last post works from, so a worker which grows is replaced before the pod is out of memory, with nothing more to set. The parts which are particular to Kubernetes are the shutdown delay, the readiness probe's failureThreshold and the absence of any preStop hook, and those are what the rest of this post is about, along with autoscaling and migrations.

Stopping without a preStop hook

When a pod is deleted, as happens to each old pod during a rolling update, two things start at the same time. The kubelet on the node begins shutting the pod down, which means sending SIGTERM to process 1 in each container. Separately, the control plane marks the pod's endpoint in the Service's EndpointSlices as terminating. Every kube-proxy, and any ingress controller, then has to notice that and stop sending new connections to the pod. Nothing makes the second wait for the first, so for a short while after SIGTERM a pod can still be handed new connections.

A server which closes its listening socket as soon as it gets SIGTERM, which is what gunicorn and uvicorn both do, will refuse those connections. The usual fix is a preStop hook which sleeps for a few seconds. The kubelet runs the hook before sending SIGTERM, so the server goes on serving during the sleep while the endpoint changes spread through the cluster. For a long time that meant an exec hook running sleep, which needs a sleep binary in the image. There is now also a sleep action, stable since Kubernetes 1.34, which the kubelet runs itself. Either way, it is something you have to know to add, and during the sleep the server has no idea it is about to be stopped. Neither gunicorn nor uvicorn has an option for this. Gunicorn's --graceful-timeout and uvicorn's --timeout-graceful-shutdown only govern what happens after the signal, by which time the listener is already closed.

Muru does this itself, with --shutdown-delay, set here through MURU_SERVE_SHUTDOWN_DELAY. On SIGTERM, /readyz starts failing at once. For the length of the delay the listener stays open and new connections are still served, but each response carries Connection: close, and idle connections are closed, so clients reconnect and land on another pod. After the delay, the listener closes, the requests in flight finish and the container exits, all within the budget, which is set below terminationGracePeriodSeconds.

To see what difference it makes, I ran a small Python client in a pod in the same namespace. It used eight threads, each on its own keep-alive connection to the Service, asking for a page which takes 50 milliseconds, for 40 seconds. Five seconds in I ran kubectl rollout restart deploy/shop. With the five second delay:

responses: {200: 6106}
errors: {} first at (s): {}
responses with Connection: close, by pod: {'shop-7f8f5b489f-bbnt5': 13, 'shop-7f8f5b489f-hl9zb': 5}

Every request succeeded, and the 18 responses which carried Connection: close were the client's connections being moved off the two old pods as each was stopped. Watching the EndpointSlice at the same time showed the first old pod being marked terminating:

shop-64df649bb5-x4hwq ready=true serving=true terminating=false
shop-7f8f5b489f-bbnt5 ready=false serving=true terminating=true
shop-7f8f5b489f-hl9zb ready=true serving=true terminating=false

That happened at 09:03:43.350. The last request that pod's access log recorded was at 09:03:43.455, and its host process didn't log receiving SIGTERM, which is when the workers start stopping, until 09:03:48.353, five seconds later. On a single node cluster, the endpoint changes reached kube-proxy within a fraction of a second.

So is the delay needed at all on a cluster this small? I set it to zero and ran the same test:

responses: {200: 6100}
errors: {'ConnectionRefusedError': 4} first at (s): {'ConnectionRefusedError': 11.4}
responses with Connection: close, by pod: {'shop-55885b469d-rxk2k': 2, 'shop-55885b469d-hftl9': 7}

Four requests failed because their connections were refused. I repeated both tests: four refused again with no delay, none again with the delay. Even when the endpoints catch up in a fraction of a second, there is a window where kube-proxy still routes new connections to a pod whose listener has closed. On a larger cluster, with an ingress controller in front, that window only gets longer, which is why the Muru documentation starts with five seconds.

Under Docker or Compose the delay defaults to zero, since a stopped container stops getting connections straight away. Under Kubernetes it is a single environment variable in the Deployment, and the pod spec has no preStop hook.

Resizing and restarting in place

As of Kubernetes 1.35 it is a stable feature that a running container's CPU and memory can be changed in place, through the pod's resize subresource, which a VerticalPodAutoscaler can also use. With the resizePolicy in the Deployment above, restartPolicy: NotRequired for both, no restart is needed, as the kubelet just changes the container's cgroup limits. Muru reads them again at its next memory check, which a pod I gave three CPUs and 1.5Gi showed in its log:

container limits changed: cpu.max from 200000 100000 to 300000 100000, memory.max from 1.0GB to 1.5GB

The memory guard then works from the new limit, and the warnings Muru gives at start are checked again, so a resize which leaves the workers short of CPUs or memory says so. What doesn't change is the number of processes and threads, which are configuration. Giving a pod more CPU without more workers to use it gains little, and giving it less memory only has the memory guard replace workers sooner. A change to the resources which ought to change the number of workers is better made to the Deployment, with the MURU_SERVE_* settings changed alongside the limits, so it rolls out as new pods, which the shutdown delay above already covers.

A resize did turn out to be a handy way of showing what happens when a container is restarted without its pod going anywhere, as also happens after a failed liveness probe. Setting restartPolicy: RestartContainer for memory in the resizePolicy instead has the kubelet restart the container for a resize. I did that and, with the client opening a new connection for each request, changed the memory limit of one pod. The container restarted in the same pod, on the same node with the same address.

The endpoint went to not ready 1.5 seconds after the resize, and nothing failed:

responses: {200: 4448}
errors: {} first at (s): {}
responses with Connection: close, by pod: {'shop-567fc9d9c7-46lpz': 59}

The reason nothing failed is the failureThreshold: 1 on the readiness probe. With a rolling update the pod is deleted, so its endpoint is marked terminating at once. A container restarting in a pod which isn't going anywhere doesn't get that. The only thing taking the pod out of the Service is the readiness probe, and by default a readiness probe has to fail three times in a row. Every two seconds, three failures take up to six seconds, which is longer than the five second delay. Taking the setting out and repeating the same resize under the same load shows what happens:

responses: {200: 4448}
errors: {'ConnectionRefusedError': 10} first at (s): {'ConnectionRefusedError': 10.0}
responses with Connection: close, by pod: {'shop-5dfdbd5847-nz5bh': 180}

The resize went in at 09:28:23.665. Muru started its shutdown delay, the kubelet's readiness probe got a 503 from 09:28:25, and Muru sent Connection: close on 180 responses. But the EndpointSlice still listed the pod as ready until 09:28:29.062, by which time the old container had exited.

A threshold of one would be a risky setting for a server which answers readiness from the same threads that serve requests, since one slow probe under load would take the pod out of the Service. With Muru, /readyz is answered by Muru itself and not by a worker, and as the last post showed it answers in a couple of milliseconds with every worker busy, so a failure means the pod really is stopping or has no ready workers.

Autoscaling on busy slots, not CPU

A HorizontalPodAutoscaler usually scales on CPU. For a Python web application which spends most of its time waiting on a database, that is close to useless. Every thread can be waiting, with requests queuing, while the CPU sits almost idle.

Muru publishes the measure that actually matters, muru_capacity_utilisation, the share of worker slots in use averaged over 30 seconds, on the admin port's /metrics. To get that to an autoscaler takes Prometheus to scrape it and the Prometheus adapter to present it as a pods metric through the custom metrics API. The pods got the usual scrape annotations:

template:
  metadata:
    annotations:
      prometheus.io/scrape: "true"
      prometheus.io/port: "9090"

and the adapter got a rule for the one metric:

rules:
  default: false
  custom:
    - seriesQuery: 'muru_capacity_utilisation{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: namespace}
          pod: {resource: pod}
      metricsQuery: 'avg(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)'

The autoscaler aims to keep the average across pods at 70 percent:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: shop
spec:
  scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: shop}
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric: {name: muru_capacity_utilisation}
        target: {type: AverageValue, averageValue: 700m}

For load, the client ran 24 threads for four minutes, each asking over and over for a page which waits half a second as if on a database. Each request was on a new connection, so the Service spread them across however many pods there were, much as an ingress controller balances each request. Two pods have 16 slots between them, two workers of four threads each, so 24 requests at a time can't all be served at once. Checking every ten seconds:

NAME   REFERENCE         TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
shop   Deployment/shop   0/700m      2         10        2          40s
shop   Deployment/shop   391m/700m   2         10        2          50s
shop   Deployment/shop   853m/700m   2         10        2          61s
shop   Deployment/shop   839m/700m   2         10        3          81s
shop   Deployment/shop   754m/700m   2         10        4          91s
shop   Deployment/shop   732m/700m   2         10        4          2m1s
shop   Deployment/shop   726m/700m   2         10        4          3m2s

The autoscaler's events:

New size: 3; reason: pods metric muru_capacity_utilisation above target
New size: 4; reason: pods metric muru_capacity_utilisation above target

Within a minute of the load starting, the Deployment had gone from two pods to four. It then stayed at four, because 24 requests over 32 slots is 0.75, and the autoscaler ignores anything within its default tolerance of 10 percent of the target. While all that was happening, kubectl top showed the pods using next to no CPU:

shop-5dfdbd5847-d5ds9   10m   39Mi
shop-5dfdbd5847-nz5bh   10m   39Mi
shop-5dfdbd5847-pjmzj   11m   40Mi
shop-5dfdbd5847-pmfdt   10m   39Mi

That is about 2 percent of the 500m each pod requests. An autoscaler targeting 70 percent CPU would have left the Deployment at two pods, with a third of the requests waiting for a slot. When the load stopped, the metric dropped to zero, and the autoscaler took the Deployment back to two pods about five minutes later, its default stabilisation window for scaling in.

The other figure worth scaling on is how long requests wait for a slot, which is the queue phase of Muru's request duration histogram. Asking Prometheus for the 95th percentile ninety seconds into the run, with its two minute window covering the time at two pods:

histogram_quantile(0.95, sum by (le) (rate(muru_request_duration_seconds_bucket{namespace="muru-blog",phase="queue"}[2m])))

gave 0.457 seconds. Unlike the utilisation, this is a single figure for the whole Deployment rather than one for each pod, so it can't be given to the autoscaler in the same way. It can instead be set up as an external metric, through a separate rule in the Prometheus adapter, or by using KEDA in place of the adapter. KEDA is a project which manages a HorizontalPodAutoscaler for you and feeds it from sources such as a Prometheus query, so a query like this one can be given to it directly, with a threshold of, say, 100 milliseconds. Either way, the Deployment would then scale on how long users wait rather than on how busy the workers are.

Neither gunicorn nor uvicorn exposes anything like this. You can get at request latency through a Prometheus client in the application, but how many slots are busy and how long a request waited for one are things only the server knows.

Migrations as a Job

In the releases post, migrations ran from the new release, with muructl command run shop@r4 migrate, while the old release kept serving. On Kubernetes the new release is a new image, and the natural place to run something once from it before the Deployment changes is a Job. The project declares the command as before:

[tool.muru.commands.migrate]
description = "Apply database migrations"
run = ["python", "migrate.py"]
timeout = "10m"

and the Job runs it from the new image:

apiVersion: batch/v1
kind: Job
metadata:
  name: shop-migrate-2-0
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      securityContext:
        runAsNonRoot: true
        seccompProfile: {type: RuntimeDefault}
      containers:
        - name: migrate
          image: muru-blog-k8s:2
          imagePullPolicy: IfNotPresent
          command: ["muructl", "command", "run", "migrate"]
          env:
            - {name: SHOP_DATABASE, value: "postgresql://db.shop/shop"}
          securityContext:
            allowPrivilegeEscalation: false
            capabilities: {drop: ["ALL"]}

With no muructl serve running in the Job's pod, muructl command run runs the command itself, using the image's interpreter, the project's environment and the command's timeout. The Deployment is then only updated once the Job has completed:

kubectl apply -f migrate-job-2.yaml
kubectl wait --for=condition=Complete job/shop-migrate-2-0 --timeout=120s
kubectl set image deploy/shop shop=muru-blog-k8s:2

job.batch/shop-migrate-2-0 created
job.batch/shop-migrate-2-0 condition met
deployment.apps/shop image updated

The migration's output is in the Job's logs:

connecting to postgresql://db.shop/shop
applied 0007_order_notes
applied 0008_order_notes_index

A failed migration fails the Job, since muructl exits with the command's status. Running the migration in a Job of its own without the database setting:

SHOP_DATABASE is not set
muru: the command exited with status 1 after 8ms

NAME             STATUS   COMPLETIONS   DURATION   AGE
shop-migrate-2   Failed   0/1           3s         3s

A kubectl wait for that Job would never see it complete, so a deploy script running the three commands above would stop there rather than update the Deployment. With backoffLimit: 0 the Job doesn't try again, which for a migration that has failed halfway is usually what you want.

You could of course write command: ["python", "migrate.py"] in the Job and get much the same result with gunicorn in the image. What the declared command buys is that the Job, a kubectl exec into a running pod and the daemon from the releases post all run the migration the same way, from the one definition in the project, with its timeout.

Looking inside a pod

Everything the last post did with docker exec works through kubectl exec, since muructl in the container finds muructl serve through its run directory. With a request stuck in a page which sleeps for eight seconds:

kubectl exec deploy/shop -- muructl requests

RELEASE              GROUP        QUEUED  QUEUED-OLDEST  THREADS  THREAD-OLDEST
shop@r1              default      0       -              1/8      1.9s

kubectl exec deploy/shop -- muructl debug stacks

muru-app-2 in worker 1:
  serving GET /slow?seconds=8 for 1.9 s (request 01M6TNH4EM7EES4KX4129GEBM8)
  File "/opt/muru/muru/_pool.py", line 30, in serve_jobs
    serve()
  File "/app/app.py", line 18, in application
    time.sleep(seconds)

Who is allowed to do this is then up to RBAC, like any other kubectl exec.

What's next

Very little in this post needed Muru to do anything Kubernetes-specific. The shutdown delay is one setting, the workers are set to go with the pod's resources, the autoscaler reads a metric Muru already publishes, and the migration is a Job running a command the project already declares.

The autoscaling here works from outside, with Kubernetes adding whole pods once the workers are busy. The next post, scaling from what the workers report, goes back to the daemon, where Muru decides itself how many workers a release needs, as a scaling policy also has it do within a pod, and where, because it knows why requests are waiting, it can tell whether more workers will help or whether something in the release needs changing.

15 Nov 2026 12:00am GMT

I review everything Claude Code writes, but at the current speed increase, it's easier to miss things. So after my review, a second, independent reviewer checks the work, and it gets more than the diff.

Here's how I've set it up.

The idea here is to use a different model/tool (in my case Codex) to keep bias out. But instead of a pure diff, which is already helpful, the reviewer also needs to know what the aim was.

For serious work (one-off scripts can get away with less), I start each feature with a grillme session to come up with a concise and well-scoped spec. I updated my variant of this skill to write the resulting spec to a local specs folder (goal, decisions (including rejected options), in/out of scope, acceptance). This way the reviewer has much richer context about the diff it's reviewing.

This is the spec template my grillme skill writes to specs/<slug>.md, kept under ~40 lines:

# <Feature>

## Goal
## Decisions
## Scope
## Acceptance
## Out of scope

If you're new to skills, check out how I use Claude Code skills for structured AI collaboration.

This was also a succinct comment from somebody on X when I posted about this last week:

"a reviewer with zero context from the writing session, otherwise it's grading its own homework. Do you give the reviewer the spec, or only the diff?"

So the answer is both!

A tight CLAUDE.md (touch only what the task needs, grep callers before changing a signature, ruff → ty → pytest) stops the simple mistakes before review.

So my flow for more serious work: grillme → spec → Opus implements → I review (!), then Codex for a second, independent and well-scoped review.

Recent example: Codex raised one P2. Each input passed validation on its own, but their product overflowed the DB column. Unlikely in practice for that app, but cross-field constraints like this are easy to miss (I've seen it in 1:1 coaching more than once).

The diff tells the reviewer what changed. The spec tells it whether that was the point.

15 Nov 2026 12:00am GMT

News

In Memoriam: Tim Peters, 1952-2026

Guido van Rossum shared the news that Tim Peters died on July 15. He wrote the Zen of Python (PEP 20) and gave Python Timsort, and as "Timbot" he answered countless questions and set the tone for early core development. Pythonistas are sharing their memories in the thread.

Wagtail CMS News

DRF vs Django Ninja: the Wagtail 8.1 API schema

Wagtail's new API runs on Django Ninja, not DRF, and Sage Abdullah and Thibaud Colas explain why: schema generation is first-party rather than deprecated in favor of drf-spectacular, and Pydantic schemas follow the type hints Wagtail already uses.

Djangonaut Space News

Picking Up Django #26355, Part 1: Understanding the Problem

Emanuel Jaimes explores Django ticket #26355, investigating previous implementation attempts and possible approaches to supporting PostgreSQL's array_append functionality in ArrayField.

Landing My First Commit in Django Core.

Goodluck Igbokwe shares his journey to his first Django core commit, exploring ImageField validation and the importance of understanding the code behind the documentation.

Python Software Foundation

Python 3.16.0 alpha 2 is now available

uuid4() and uuid7() now have C implementations and run 15-30x faster, re picked up nested character sets, and startup and garbage collection are quicker. Alpha 3 follows on December 15.

PyPI's redesigned project pages with clearer security signals are now live

Each PyPI project now has a Security tab that shows whether a release has full provenance, meaning every file has an attestation, or none. The redesign also adds search and filters to the release files list, and shows file counts and yanked or quarantined status in the release history.

Joshua Mcshannon Joins the PSF as Infrastructure Engineer

The PSF's new Infrastructure Engineer will spend his time cutting operational toil across python.org, PyPI, and the systems behind PyCon US. He has already started on Kubernetes cost savings.

Updates to Django

Today, "Updates to Django" is presented by Raffaella from Djangonaut Space! 🚀

Last week we had 17 pull requests merged into Django by 15 different contributors - including 8 first-time contributors! Congratulations to ines besrour, himanshu0870, ShabirAhmadLinux, Patel Namraa, Dmitry, Sahil Advani, Kunal Kumar and Hossein Shams for having their first commits merged into Django - welcome on board!

New in Django 6.2:

• An input_type of "textarea" was added to the Textarea widget. Like other widgets, when set to "hidden" it now renders the hidden HTML attribute. (PR #21857)
• django is now the preferred name for Django's command-line interface. django-admin remains available as an alias. Both names provide the same functionality. (PR #21260)

New in Django 6.1:

• Fixed a bug in Django 6.1 where accessing a deferred field with the FETCH_PEERS fetch mode overwrote values that peer instances had already modified locally or loaded via refresh_from_db with an explicit fields argument. The fix also addresses a KeyError when a peer's row had been deleted (#37436).

Django Fellow Reports

Django Fellow Report - Jacob

Between release blocker and security work, Jacob ran a small Trac sprint over Sarah's triage workflow changes and this year's Google Summer of Code dark mode project, and followed up with the GDAL maintainers on findings from the security releases. The main management command is now named django.

Django Fellow Report - Sarah

A four-day week spent mostly on triage and the "Ready for Checkin" backlog, accepting tickets on StepValueValidator crashing for large numbers, order_by("pk") on a union of composite-pk models, and NegatedExpression mishandling FullResultSet. She also enforced ticket assignment limits and triage restrictions in Trac.

Sponsored

Prod-like data for your coding agents

Coding agents write Django code against fixtures and guess what your data looks like. Baseshift gives every agent, PR and developer its own masked copy of your production database, ready in seconds, with no infrastructure changes. Start free and point DATABASE_URL at your first clone.

Articles

Django's Generic Views: FormView

Ben Cardy's fourth class-based view post walks through FormView, which renders a form, validates it, and redirects on success.

Making Django admin easier to use every day

Some examples of a team customizing the admin as needed for their specific use cases. A worthwhile effort if you use it heavily.

Django: introducing django-launch-editor for jump-to-file debugging

Adam Johnson's new package makes the file paths on Django's debug page clickable, opening your editor right at that line. When the editor is out of reach, in Docker or on a server, clicking copies the location instead.

Authenticating Django Channels WebSockets without putting the JWT in the URL

A browser cannot send an Authorization header on a WebSocket handshake. The common fix puts the token in the query string, where every access log keeps a copy. This is the other way, and the four places it can quietly go wrong.

Using OpenTofu to manage your Infrastructure as Code

Ryan Cheley finally learned the OpenTofu that Django Commons runs on by importing his own AWS, Coolify, DigitalOcean, GitHub, Hetzner, and Tailscale setup.

Lock your GitHub Actions

Action tags can be moved, so a workflow pinned to a safe-looking tag can start running an attacker's code. Hugo van Kemenade walks through GitHub's new workflow lockfiles.

Catch the import. Name the culprit. Keep threads free: Introducing gilguard

gilguard is a tiny, zero-dependency package for free-threaded Python (3.13t, 3.14t, 3.15t). Import it first. If any later import quietly turns the GIL back on, you get an error naming the extension module, the package that shipped it, and the line of your code that pulled it in.

What makes uv slow?

uv maintainer Konstin walks through where uv still spends its time, from source distribution builds and network requests down to TOML parsing (about 30ms just to read Airflow's lockfile), hashing, NTFS, and memory allocation.

Events

Wagtail Space 2026

Wagtail Space is next week, November 18-20. The event is 100% online and you can watch live or watch the recordings at your convenience. Expect talks from Wagtail and Django experts plus guest speakers on AI, sustainable tech, and other trends shaping the web. Tickets are free, so grab one before Wednesday.

Videos

People First, Code Always - Kenneth Love

Longtime Django developer Kenneth is the guest in this episode from Jon Gould at Foxley Talent. Lots of insights from coding in the pre/post AI world.

Databases, Django APIs, and AI - Haki Benita

The video cut of Django Chat #209, just over an hour with Haki Benita on getting more out of the ORM and PostgreSQL indexes, where Django Ninja fits, and what Django still needs for typing and APIs. That last part is the ground DEP 0017 covers, adding content type parsing and a request.data attribute that handles JSON, and the forum thread has the wider debate.

Django Job Board

The PSF's CPython Developer in Residence opening is still accepting applications, alongside senior Python roles at Manychat in Barcelona and Proxify, plus a founding engineering role at Alloy.

CPython Developer in Residence at Python Software Foundation

Senior Python Engineer (Barcelona-based / Relocation support to Barcelona) at Manychat

Senior Backend Developer (Python) at Proxify AB

Founding Forward Deployed Engineer at Alloy

Projects

prafulgulani/django-experimental

A playground for trying experimental Django features behind feature flags, installed as django.experimental.

swarfte/django-ninja-expose

Add @expose() to a Django model. Get a REST API.

Sponsor Django News

Reach 4,300+ Django developers every Friday. See sponsorship details and rates.

Planning something for the holidays? Reply and ask about our Black Friday deal.

13 Nov 2026 5:00pm GMT

I've made another package! django-launch-editor makes the file locations on Django's debug error page into links that open your editor.

Here's how it looks, on a KeyError from the package's example project:

(Click to enlarge.)

The change against stock Django is subtle, but can you spot the underlines on the file locations? They're now links that open your editor at the right line, and even the right column when Python knows it.

I think this is gonna be a neat time saver. Before, I would normally remember the filename and line number, then go open that in my editor. Or I would fiddle around copying the file path to then open it. Now it's just a click!

Setup is as easy as installing the package, setting Django's DEFAULT_EXCEPTION_REPORTER setting to the package's reporter class, and adding its URLs. See the README for the details.

When DEBUG is False, the reporter adds nothing to error pages, and the URL isn't even registered, so there's zero overhead in production.

When no editor is found

django-launch-editor launches your editor from the Django server process, so it can only launch an editor directly when you run Django on the same machine as your editor. If you run your development Django server elsewhere, such as in a Docker container or on a remote server, it can't find your editor. In that case, clicking a location shows this dialog and copies the location to your clipboard instead:

That way, you can still paste the location into your editor's "go to file" prompt, without fiddling around selecting text.

from launch_editor import LaunchEditorError, launch_editor

try:
    launch_editor("src/example.py", line=12, column=4)
except LaunchEditorError as exc:
    print(f"Could not open editor: {exc}")

$ python -m launch_editor --guess
code

$ python -m launch_editor example/views.py:20:15

$ uvx --from launch-editor-py python -m launch_editor --guess

08 Nov 2026 6:00am GMT

News

Django security releases issued: 6.1.3, 6.0.10, and 5.2.19

Six CVEs, three of them in GeoDjango: two server-side request forgery holes where GDAL resolved URLs or files from GeoJSON CRS members and SpatialReference strings (full protection needs GDAL 3.9+, flagged by the new gis.W001 check), plus log injection through GeometryField. The rest cover base64 uploads that ignored FILE_UPLOAD_MAX_MEMORY_SIZE, cache keys that confused missing and empty Vary headers, and password hashes shown to view-only admin users. Upgrade soon.

Django Software Foundation

Django Steering Council Meeting, November 2

The Steering Council met with the Fellows ahead of the mid-January feature freeze, with Carlton Gibson aiming to land the async transactions PR before then. They also weighed what to do now that sqlparse is no longer maintained (it backs --debug-sql and inspectdb), how to surface DEP discussions on the Forum, and GSoC 2027 ideas.

Python Software Foundation

Announcing new Security Developer at the PSF

Brett Smith, formerly of the W3C and Software Freedom Conservancy, joins Seth Larson and Mike Fiedler on the PSF security team in a new role funded by Anthropic. The Python Security Response Team is handling 3-4× last year's volume of reports and CVEs, and PyPI malware is up 60%, so Brett will help with remediation and scale PyPI's autonomous quarantine program.

How the Python ecosystem responded to the eMudhra intermediate compromise

Within two days of the October 20 CA compromise, fixes shipped for certifi, Python, pip, uv, sigstore, virtualenv, and more, and Python added a last-resort denylist for the bad intermediates (CVE-2026-115557) that was backported to older releases. Seth Larson explains why certifi, pip, and Python sat in the critical path, and why browsers were safe from day one thanks to certificate revocation lists, which Python may adopt.

Wagtail CMS News

Wagtail 8.1: automate more with an improved API and agent-ready tools

The preview REST API can now write more content types, expose StreamField block schemas, and drive moderation workflows (submit, approve, reject, cancel), alongside official agent skills and a prototype CLI. Editors also get SEO checks for meta titles and descriptions, pages can be served as JSON or Markdown from the same URL, and multilingual sitemaps now include translated pages.

An official headless demo for Wagtail

bakerydemo-headless rebuilds the bakery demo as a fully decoupled Next.js front-end in TypeScript, with live preview still working in the CMS. Thibaud Colas notes it runs about 95% on Wagtail's built-in v3 API, whose OpenAPI schema generates typed clients down to individual StreamField blocks, with no GraphQL required.

Updates to Django

Today, "Updates to Django" is presented by Hwayoung from Djangonaut Space! 🚀

Last week we had 17 pull requests merged into Django by 12 different contributors - including 2 first-time contributors! Congratulations to Alex Henman and Spoko Dev for having their first commits merged into Django - welcome on board!

News in Django:

• Django now supports the HTTP QUERY method, allowing safe, idempotent requests with query content in the request body (#37232).
• Django templates now support template tags spanning multiple lines, making tags with several arguments easier to read (#35899).
• Fixed a regression where BoundField.legend_tag() could add a custom required CSS class more than once when passed attrs (#37463).

Django Fellow Reports

Django Fellow Report - Jacob

Jacob Tyler Walls finalized the patches and sent prenotifications for Django's November security releases, then started drafting patches for the next round. He also fixed attrs mutation in legend_tag(), opened a follow-up optimizing lookup preparation for GeoJSON geometries, and sat in on Executive Director interviews.

Django Fellow Report - Sarah

Sarah Boyce accepted five tickets, including consistent database selection for subqueries, dropping GIS support on 32-bit Windows, and a parallel test runner crash, and reviewed fixes for XOR join promotion, select_for_update() after combined queries, and warning assertions. Her own PR allowing multiline template tags was merged, and she worked on three security reports and interviewed final candidates for Executive Director.

Django Fellow Report - Natalia

Natalia Bidart prepared the branch that bootstraps Django 2028, the first calendar-versioned release: it drops Python 3.12 and 3.13, removes the EMAIL_* settings in favor of MAILERS, and finalizes several ORM deprecations, and it lands in mid-January once the 6.2 branch is cut. She also pushed docs PRs that give local builds a new theme and better accessibility, and joined three Executive Director interviews.

Sponsored

Django for Beginners, 6th Edition

The sixth edition is out now, available in digital and paperback formats. It features two new chapters, completely updated for Django 6.1+, and more. Read the first four chapters for free to try it out.

Articles

Soft delete in Django without making a mess

Stein Ove Helset builds soft delete on a deleted_at timestamp and works through where it breaks: bulk QuerySet.delete() still removes rows, unique=True blocks re-creating a deleted record (a conditional UniqueConstraint fixes that, though MySQL and MariaDB ignore the condition), restores can collide with newer rows, and CASCADE never fires for related objects. A filtered default manager paired with an unfiltered base manager keeps related lookups working.

N+1 queries in Django Admin

If a related model's __str__ reads another foreign key, every option in an admin dropdown runs its own query, and select_related() on the edited object doesn't reach those options. Override formfield_for_foreignkey() and formfield_for_manytomany() to add select_related() to the querysets behind the widgets.

Django's Generic Views: TemplateView

Ben Cardy's series on class-based views reaches TemplateView, which needs only a template_name. He uses extra_context for static values, get_context_data() for computed ones, and template_engine to render a single view with Jinja2.

Taking Website Screenshots with Django, Celery, and Playwright

Nik Tomazic builds a Dockerized Django app that takes full-page or viewport screenshots with Playwright. Each capture runs in a Celery task so the form returns right away, and the page polls a small JSON status view until the image is ready. An optional last section moves the browser to Browser Use's paid hosted Chromium for when local browsers eat worker memory or hit bot detection.

Looking for a job? Start a blog.

Tim Schilling argues that a blog shows a hiring manager how you communicate before they ever talk to you, and suggests starter topics like a recent technical decision or how you review code. He says the host doesn't matter (GitHub Pages, dev.to, and Medium all work) and asks you to write it yourself instead of with AI, since the point is to show how you think.

How Caktus Runs and Uses Local Models

Tobias McNulty and Colin Copeland describe the two-GPU workstation Caktus uses to serve open-weight models over Tailscale, so client code stays on its own network. A dense Qwen3.8-27B handles planning and long sessions, and a Qwen3.6 MoE with 3B active parameters runs everyday work at 200-250 tokens per second, both on vLLM in Docker Compose. The post includes their benchmarks, why Gemma 4 31B didn't make the cut, and the pi.dev and VS Code configs they use daily.

Per-Object Permissions in Django That Roll Back with Atomic()

Joshua Fields wrote pemba, a Zanzibar-style authorization library that keeps per-object permission relationships in your own Django database. A grant is just rows in the current transaction, so it commits or rolls back with the data it protects, which a separate service like SpiceDB can't guarantee.

Artisanal Podcasting in the Age of AI

Will Vincent has hand-edited Django Chat in GarageBand for nearly seven years, even with AI tools that promise to handle the whole thing. Editing gives him a second listen to each conversation and a kind of deep focus he no longer gets when coding with a handful of AI agents.

Caching at Scale in Django: Lessons from Production

Johnny Metz's team handled about 20x traffic growth on a multi-tenant Django API by caching aggressively: per-user lists of document IDs paired with one shared entry per document, seven-day TTLs with invalidation on Django signals, a cached identity lookup, and failing open when Redis goes down. During the rollout, database queries per second halved, and Redis now serves most reads at a 97% hit rate.

Events

DjangoCon Europe 2027 Call for Participation

DjangoCon Europe runs February 17-21, 2027, in Innsbruck, Austria, and wants talk and workshop proposals on technical and non-technical topics from new and experienced speakers. Proposals close November 15, 2026.

Podcasts

Django Chat #209: Databases, Django APIs, and AI - Haki Benita

Haki Benita talks about getting more out of Django's ORM and PostgreSQL indexes, why Django Ninja appeals for APIs, and what Django needs for typing and a built-in API story. They also get into how AI is changing the way developers learn and share what they know.

Videos

From librarian to Djangonaut: My Open Source journey with Annabelle Wiegart

In a guest talk for Djangonaut Space Session 7, Annabelle Wiegart traces her path from librarian to Django contributor, PyLadies Zurich organizer, and Djangonaut Space captain. She talks about working on ORM tickets and nullable expressions with mentors, and how contributing went from intimidating to something she could do.

Django Forum

A Path to Modernizing APIs in Django (DEP 0017 Content Type Parsing)

Picking up from the Haki Benita episode of Django Chat, Will Vincent lists what stands between Django and a built-in API: a modernized HttpRequest through DEP 0017, a pluggable serialization layer (the hardest part), and then the view layer. Carlton Gibson wants those primitives updated first, while Haki would settle on an official API interface and tune performance later.

Django Job Board

The PSF's CPython Developer in Residence opening leads the board this week, alongside senior Python roles at webbyfox consultancy and at Manychat in Barcelona, with relocation support.

CPython Developer in Residence at Python Software Foundation

Senior Python Engineer at webbyfox consultancy

Senior Python Engineer (Barcelona-based / Relocation support to Barcelona) at Manychat

Projects

amureki/django-cmdbar

Adds a Cmd+K / Ctrl+K palette to the Django admin for jumping to models, searching records (user alice), or opening one by primary key (user#42). It respects admin permissions and needs no frontend dependencies, middleware, URLs, or migrations: just add django_cmdbar above django.contrib.admin in INSTALLED_APPS.

Metron-Project/izaya

An htmx-powered autocomplete widget for Django forms that ships no required styles, so it fits any CSS framework. Register a ModelLookup with your search_fields, drop LookupWidget into a form (with multiple=True for multi-selects), and use rank_fields to put the closest matches first; it needs Python 3.14+ and Django 6.0+.

Sponsor Django News

Reach 4,300+ Django developers every Friday. See sponsorship details and rates.

06 Nov 2026 5:00pm GMT

Twisted was originally the network component of a text-based role-playing game engine. In the late 90s when it was first developed, the idea of a video game being able to send you actual emails and instant messages¹ seemed pretty neat, rather than the valence that it would have today (nightmare). We never ended up making an actual game, so we eventually removed the "game engine" piece, and just kept the protocol implementations of the email and the AOL Instant Messenger, and the DNS and eventually SSH and so on. The residual pieces of that project have not seen maintenance for a decade.

So it's not really a way to make a text-based game any more.

Then, Twisted was the premier longest-lived and best-maintained event-driven I/O library for Python. If you wanted to make high-performance servers, it was the best and most obvious way to do it. Even if it still acquits itself relatively well in this regard, its claim to fame, and to indispensability, has been somewhat obviated by the inclusion of asyncio in the standard library. Even if it has some stuff to recommend it - better-than-average testing facilities, for example - you can get better-documented and better-integrated low-level asynchronous networking tools without even installing any packages.

So it's not really the premier choice for an event-loop these days.

Twisted does still contain an SSH server and client, an email server and client, a DNS server and client, a web server and client (now including websockets). But you can cobble these together by installing a variety of packages from other sources.

So... it's a bunch of protocols? And I guess it doesn't use AI? But... why would anyone care about that? Why contribute to such a thing?

What is Twisted doing, and who is it doing it for? What is the political project of Twisted, nigh on 30 years into its existence?

Most of all, why should you care about it, or want to contribute?

If you're asking what you could use Twisted for today, it's an excellent choice if you're implementing a new network service and you need more than one protocol. If you need to do HTTP, DNS, packet capture, and a custom SSH server, it would be very hard to arrange a collage of other Python packages to manage this. It's still much better at deterministic testing than most of the standard library. It's a bit easier to hook into a GUI.

But that's not what Twisted is doing.

What I want Twisted to do for the world is still an unfinished project. It's honestly kind of bad at doing what I want it to, but it could be great. The overarching goal of having and maintaining all this stuff is to, as much as possible, demystify networked software operation and let you build Internet facing systems at a human scale.

It's all in a memory-safe language with good error handling, so you can avoid the scariest security pitfalls. It's all integrated into a single paradigm with built-in tools for testing. It's designed to scale down to be runnable on the average laptop, and not require large elaborate server farms or distributed clusters to run. Via the twist command line utility, it provides a Swiss army knife of simple tools that you can use out of the box.

In a world where all of our compute is in massive distant opaque datacenters, where a tiny cartel of gigantic tech companies control nearly the whole supply chain, one of the few things that you still can have agency over is your own server on the Internet.

It's still possible to get a domain name for a few bucks a year, a free-tier or very low-cost VPS, then stick a glue record in and run your own DNS. From that point forward, you can run your own email, your own web server, your own terminal server. But doing this requires deep expertise, and that expertise requirement is massively inflated by the operational concerns that go with running twenty different pieces of software from twenty different projects, probably in twenty different languages, with completely inconsistent aesthetics.

The reason that Twisted is event-driven is because as much as it can be a pain to wrap your head around async/await (or worse yet, Deferred directly), it is much easier for a single person to understand the concurrency requirements of an async I/O multiplexed system than to understand what is going on with an application that requires multiple thread pools. It is possible to understand concurrent code that manages parallel tasks in that idiom. The reason that Twisted integrates so many different protocol implementations and attempts² to unify them with a common authentication layer is to reduce the overhead required to manage a comprehensive suite of network services.

Even the not-using-AI thing is not an aside due to some accidental personal preference about tools - it's a core part of building an engine that you can personally understand if you dive into it, that's built with a consistent style and layered in a way that you can take apart if you need to. It's not built with AI because we're not building for AI, we're building for people and the entire point is that people are supposed to be able to understand it.

We used to have a tag line, which is somewhat less emphasized these days because its meaning is not immediately clear without this background, but was once quite prominently featured on the home page: "Twisted: Building the Engine of Your Internet". That's what I want to bring to you: an engine for you to build your Internet, that is small enough that you can make it personal, that doesn't require massive investment in management tooling just to get started.

Now, for various reasons, I no longer use twisted.names to host my own DNS (although I did for many years!), and we never wrote enough of a twisted.mail suite for it to be usable for email hosting on its own. But, if you're reading this article, it was probably delivered to you directly via Twisted's own twisted.web HTTP server, using twisted.protocols.tls to configure our own OpenSSL TLS termination. I have run with extremely high reliability for years and next to zero management overhead, in a single simple docker container. Which I then administer and upload files to via Twisted's own twisted.conch SSH server.

If that seems like an appealing project and you'd like to volunteer to help out, drop me a line and maybe I can point you at some tasks to get started. Twisted falls far short of this aspiration despite 30 years of work, and we could use all the help we can get.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

05 Nov 2026 10:51pm GMT

Creative Work

Writers went on strike to get protections against "AI". Thousands of artists have signed open letters in protest of "AI". There are so many copyright lawsuits from creative industry groups against AI that there's a whole dedicated website for it. Popular YouTubers absolutely hate it. If they're also musicians, they REALLY hate it. Across all creative industries, there is a concerted push to reject this technology.

Yet, almost unique among creative fields, many experienced programmers remain convinced that it's fine to use "AI" for programming. We do seem to hate it, and what it's doing to our industry. It's making us all miserable, but we are using it anyway.

A lot of the justification of this resignation seems to be to be because programming is not Art. If the tool can get the job done, and the job is just functional, then why does it matter?

It does matter, though. It matters because we shouldn't be using AI to produce Art, and programming is Art.

Art can be Mundane

Some people will say that programs cannot be art because programs are functional, rather than being expressive. Programs are mundane whereas art is transcendent.

This is based on a distorted understanding of what Art actually is.

In John Berger's "Ways of Seeing", he names this type of distortion "mystification". His example of this process is both amusing and illustrative. I encourage you to read it in its entirety.

In summary, though: Berger critiques the florid prose of an art historian describing a commissioned group portrait, including phrases like "subtle modulations of the deep, glowing blacks" and "harmonious fusion". The portrait is described as sublime, in nearly ecstatic terms.

Berger reveals that the reality of this portrait is that a poor old painter needed some work, and some officials probably thought it might be nice to have an official portrait. So they paid some money to the poor old man, and he painted it, and then they had a painting. It's a well-executed portrait of a group of people. Beautiful, even. But it was work commissioned for a fairly mundane purpose and it suited that purpose just fine. It was not, and is not, a divine relic.

Culturally, we are prone to mystifying painting, and sculpture, and film, and music. We imbue them with "subtle modulations". We ignore their functional aspects - we desire decoration, amusement, and distraction - and focus on their emotional impact.

Don't get me wrong: I love me some good aesthetic philosophy. I think it's great to really examine our reactions to artwork and to try and gain a deeper understanding of our culture and our selves through media analysis. If anything we really need to do more of it.

This does not mean that the creation of such works is mystical or that it should be venerated beyond any other sort of labor.

Not least of which other types of labor that are adjacent to, but not as culturally venerated, as fine art. We tend to mystify the work of a novelist, but to denigrate the work of a journalist. In reality, the functional prose of the journalist is no less important and deserves no less respect.

Although they might be far below the ethereal realm that novelists inhabit in our collective imagination, even journalists receive more respect and thus more mystification than lowly copywriters. Yet, there is no transcendental distinction between "novelist" and "copywriter"; the many of the skills are the same, and the distinction is merely an accident of commerce and opportunity.

In fact, many famous writers have famously inhabited both roles. This is not an accident! Working with words professionally, even (perhaps especially) mundane words, is excellent practice for working with words in a more purely artistic context, because even mundane creativity is still artistic.

Code can be Beautiful

One thousand Internet years ago, when I was in my late teens, I would describe myself as a "code poet". I was relentlessly mocked for this as what the Youth would today call "being cringe", and at the time was referred to as "pretentious".

I succumbed to the peer pressure, removed it from my email signature and my bio. While I still believed strongly in the parallels, I accepted that - socially, at least - comparing code to poetry, or indeed to Art, was a silly thing to do.

However, I never abandoned the idea, in my heart.

The thing that I am most well-known for, the invention of Deferred, was specifically an aesthetic reaction to the tedium of passing callback and errback parameters to every single remote procedure call in an RPC client/server application. Those two callbacks got the job done just fine. But they were ugly, and annoying to work with.

Deferred is an intentional poem about asynchronous task execution, with a deliberate eye to the aesthetics of the problem and the experience of using it. It was influential because of its focus on aesthetics.

I do not want to overstate the beauty or profundity of this minor contribution, or indeed its durability. That a poem exists does not mean it is a great poem, merely that it is a poem.

Our aesthetic culture around programs is more like folk epic poetry than fine art, so the influence of this contribution is less about its specific enduring power than it is about its influence on what came next; from MochiKit.Async to JQuery Deferred to JavaScript Promises and eventually to async/await; a long chain of different artisans each adding something of their own until the original has all but dissolved. (And I wasn't the "original" here, either, as I drew heavily from the E language's Promises, among other things.)

In order to make code into a deliberate artistic expression, one must have spent quite a bit of time contemplating the problem domain. Without having experienced the tedium of manually passing a thousand callback parameters, I would have had neither the skill, nor indeed the motivation, to bother creating such a thing.

Now, most code does not have to be like this. Most code does not get to be like this. Most code is functional, workday code. Most code could not make a lady weep. It's just copy-writing, if you will.

As I explained previously, most writing couldn't do that either. Most writing is just copy-writing, too. Most visual art is advertising. Most live music performance is background music in bars that will go largely ignored.

However, code that is intentionally aesthetically designed tends to be important, both socially and technologically.

We do have some tradition of self-mystification in software. As Abelson memorably put it, "Programs must be written for people to read, and only incidentally for machines to execute.", so we have long had some conception of programs as highly expressive, even if we can't always agree on what they're expressing or to whom. We will occasionally wax poetical about the philosophical implications of a particular piece of software. This is not unique to a single piece of software, either; more than one community has indulged in similar philosophizing.

The expressive and aesthetic qualities of software are not limited to reading source code or interacting with other programmers via APIs, either. For example, every year, Federico Viticci does a review of Apple's new operating system, which is (among other things) an aesthetic critique. Such a project would not be possible if the software did not have an aesthetic impact on its users.

Not to mention that every video game review is also a software review.

Defend The Mundane

If we use AI to erase all the copy-writing, all the graphic design, all the boring mundane art, and yes, all the boring custom WordPress theme development, then we will be removing all the practical opportunities for the vast amounts of practice and contemplation required for people to elevate their craft to eventually achieve great things. Education is great, but the majority of true skill development happens on the job and always has.

This doesn't mean that we can't use abstractions, or automation, to make our work easier. Programming is the art of abstraction, of understanding how to compose smaller ideas into bigger ones, of how to understand the automation of a larger system by understanding the rules that automate smaller ones and understanding how to combine them.

When we use "AI" to eliminate that understanding rather than raise it up to a higher level, to entirely destroy that creative decision-making process, we do a disservice both to ourselves as programmers and to our users. We would be doing a disservice to our users and our downstream fellow developers in the same way that a visual artist would be doing a disservice to their viewers or a musician would be doing a disservice to their listeners if they served them auto-generated filler instead of their own creative output.

Slop is slop, no matter the medium.

Each mundane project has some tiny chance - let's say, something like 0.1% - of achieving greatness. If we do a single project with AI, then sure, whatever, there's almost no chance that that project was going to be the one hit to create that career-defining moment for an engineer working on it. If we make a habit of doing all projects that way, though, we take the total likelihood of those moments of greatness from "definitely sometimes" to "never".

The precisely appropriate ways in which to resist AI encroachment on all software development lie well beyond the margins of this one short post. How much you can resist and which specific uses you should resist are up to you. But it is worth resisting in software just as much as it would be worth resisting in any creative medium.

Programming isn't special. It's just Art, and Art is the most human - and thus, the most universal - thing that there is.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

09 Oct 2026 7:00am GMT

One of the issues that I have with the current generation of "AI" products is that they do not appear to take their own premises seriously. I look at a plethora of obsequious chatbots claiming to be serious tools for problem solving, and I think, this is not what a problem-solving tool would look like.

Even before we get to the tremendous ethical problems with the frontier labs, it is this impression of their composition as a product that makes me feel, constantly, whenever I am interacting with them, that they are less a software product than that they are a grift, a scam designed to make me feel like I am interacting with a product that has capabilities that it simply does not, to try to lull me into a false sense of security that I can trust it.

The frontier labs are of course the worst offenders, but every criticism here applies just as much to Ollama, which (if anything, due to the obviously poorer quality of the available models themselves) needs these features even more than the frontier labs do.

Here, I will set down a few features that might convince me that an LLM-based product, particularly one focused on research or software development, was actually serious about helping me do useful things with it.

Make "Checking For Mistakes" A First-Class Feature

This is the biggest issue, and the major reason that I was inspired to write this post.

It is a truth universally acknowledged, that AIs cannot reliably provide information.

I could cite a ton of news articles and studies about this fact, but there is no need. Every single chatbot admits this, up front, in a fine-print disclaimer as a core part of their user interface. Gemini says "AI can make mistakes, so double-check responses", Claude says "Claude is AI and can make mistakes. Please double-check responses.¹" ChatGPT says "ChatGPT can make mistakes. Check important info.".

Every time I see that last one, I wonder how I'm supposed to know what "info" is supposed to be "important".

All of these warnings are all small, gray text, painfully obviously included as legalese to push responsibility back onto the user rather than to help with anything. This is a core limitation of all these products. Checking their output is a part of the workflow for using them that:

1. you absolutely cannot skip or skimp on without creating risks to yourself and whoever you are conveying its output to, and,
2. it is very easy to skip or skimp on and you are encouraged at every turn to do so, because "just trust the output" is one of the quickest ways to save time.

A chatbot product that took this weakness seriously, as an actual consideration for using it, would put a checkbox next to every claim in its output. It would be a 2-column worksheet, where you've got the LLM output in the first column, and next to it, human notes in the second column, explaining what work went into checking this claim, and a big checkbox that you would only check off after you believe you'd checked its claims thoroughly enough.

Coding assistants would need to have some version of this as well. Right now, this is pushed off into code review, which means it is a dark pattern which subtly encourages the "author"² to offload this work to their code reviewer without ever looking. Once again, "it's probably fine, I don't need to check" is the quickest way to save time and churn out those PRs faster.

It might even be useful for coding harnesses to have some affordance for checking code before it even runs tests. As the vendors themselves have admitted, it's not just expensive to burn tokens on your "AI", you also end up burning far more compute on the AI. Being able to check your diffs before sending them over to uselessly exhaust your testing compute cluster would be useful.

If your product tells me that it makes mistakes and I must be the one to check for the mistakes, but then gives me zero tools to check for mistakes, I cannot take it seriously.

More Citations to Check, And More Details

Most chatbots prefer to give an answer, rather than a citation. In my own personal use, I find that when asked to provide a list of citations with clearly marked sources for each one, they will appear to "get bored" halfway through the list and simply stop including citations at some point.

When the bots include citations at all, present them as inline annotations that say nothing but the domain name of the search result, in a font so small that it's barely legible, and an equally indecipherable icon that is fewer than 16 pixels on a side.

This is backwards.

Now, I am aware that these citations do come from somewhere, and in an attempt to reduce hallucinations, all of the major providers support some form of "grounding"³, and that those little barely-readable citation links are referencing actual structures in the RAG pipeline and not just potentially-hallucinated tokens, but I'm not talking about the underlying machinery in the model, I'm talking about the presentation to the user.

Plus, regardless of whether a snippet of text came from a RAG query, we know that LLMs can never provide an authoritative result; it's a fundamental limitation of the technology. They can still garble the results of RAG as much as they can misrepresent any other training data. This means that it must never present its results as authoritative.

If you ask an AI to do research queries, every result should be presented as a list of citations. Moreover, the presentation should display each citation as a large object of in its own right, with clearly identified metadata, including not just the site where it was found but its publication date and, if possible, the name of the author. The literal, unmodified quotation (not from RAG, not a summary: a quotation extracted with a regular program and not an LLM) should be front-and-center, larger than any AI-generated text.

If the AI product wants to editorialize or summarize (which should not always be necessary!), the AI-generated text should be presented as small text underneath the citation that has been found, de-emphasized as much as the disclaimer is right now, at the very least until the user has verified that the summary is accurate. Perhaps, for a research project, a "did you read the citation" checkbox might even be helpful.

If your product openly tells me that it will scramble, misrepresent, or omit its citations in its summaries, and I must read the original human-authored citations to be sure, but then gives me no tools to track my reading of those citations or even any way to find them, I cannot take it seriously.

No First-Person Output, No Apologies

There is no reason for a software development or research tool to use first-person language to describe itself. They should not do so. In fact they should not be allowed to do so.

There is also no reason that they should ever apologize. It is a waste of everyone's time; it's a waste for the chatbot to generate the apology, it's a waste for the user to read the apology, and it's a waste for the user to respond to the apology. Yet they unfailingly do this upon every correction.

The vendors of these tools know that they are routinely causing mental-health crises. In response, they have added non-functional "guard rails" that can still, in 2026, easily be bypassed.⁴

A product seriously interested in helping with productivity would correct this glaringly obvious flaw, focus on the task at hand, and stop emitting useless verbiage.

In the previous two sections, I tried to focus on ways in which the harness would be constructed differently even if the LLM technology is fundamentally impossible to improve; in this case, I have to assume that the labs have some control over the model itself. But unless they are truly incapable of influencing their output (and all their "benchmarks" and "capabilities" seem to indicate that they can control it very tightly) they ought to be building models that are much less verbose.

More Non-Natural-Language User Interfaces

Although natural language could hypothetically be a powerful interface for interacting with a computer system, the practical upshot of LLM natural language interfaces is that these interfaces are imprecise and repetitive, full of superstitions masquerading as "best practices". The inputs are a mess and the resulting outputs are a mess.

The general way of addressing this unstructured mess is to allow the chatbot to directly take action in response to the user's input; in other words to supply it with "tools" via an MCP server. But again, this is backwards. If we cannot even express our intent clearly in the first place, why are we trusting this system to take potentially destructive and harmful actions on our behalf?

Instead, I would expect a product that was seriously invested in helping me accomplish specific tasks, to have user interfaces specific to those tasks. Is it supposed to be able to be a security scanner that can discover OWASP top 10 bugs in a codebase? Have a button for that. Build that functionality into your harness, train it directly into the model, use smaller models that can satisfy that functionality more effectively than throwing it at the planet-sized brain of Fable or whatever.

I'm aware that there are small software startups that do something like this, but they are bolted on to the side of the main model providers' APIs, not integrated into the core of the product and not using their own models and AI systems to achieve consistent and repeatable results.

Strong Data Provenance Indicators

Chatbots produce data tables pulled from websites, from APIs, from MCP tools or from summarizing and scrambling the user's input. In order to provide the illusion of a seamless interface, this data is presented in-line regardless of where it comes from. But some of these outputs are produced mechanically via regular old API calls, for example, from the result of calling a tool or querying a website, but presented uniformly.

But there is a huge difference between an authoritative data source being inlined as part of a chatbot conversation, being treated as input by the chatbot, and some ad-hoc hallucinated data being treated as output of the chatbot.

If a product is trying to help me make accurate, empirically-grounded, data-driven decisions, the source of the data is critical.

Integrated into the "check for mistakes" and "verify citations" workflow I described above, there's a necessary "verify data programmatically" pass as well; to have tools that will treat portions of the output as a regular spreadsheet, allowing regular-old computer arithmetic to verify things and showing where such arithmetic was used, and how.

Better User Control of Reproducibility

Anyone familiar with the technical specifics of LLMs will know that they have a variable called "temperature" which controls the degree of randomness that the LLM uses to produce its outputs. But most users don't know this, because it isn't exposed as part of the user interface by default.

This leads to a subjective impression that you asked ChatGPT, and you got ChatGPT's authoritative answer.

You can't just set the temperature to zero and still get useful results - I am aware that it does more than just scramble the output at random, and there are perhaps good reasons that simply exposing just a temperature setting would not be that useful to users. But if we followed some more of my earlier recommendations for making more structured UI elements to solve specific problems rather than having long back-and-forth chats where each refinement depends on the previous response, perhaps those elements could also re-play the process so that users can see how reliable the bot is at a particular task and develop a sense of how the stochastic nature of the process actually affects it.

Similarly, if a user is trying to solve the same problem repeatedly with a chatbot, and the chatbot product has numerous computational tools that don't really have anything to do with the LLM, such as deterministic data-processing tools, then having a way to freeze the non-deterministic parts of the transcript but re-populate a particular data frame with updated information and fork / continue the conversation from there would be a way to avoid introducing pointless additional randomness when you already know what tool you're trying to use.

The fact that every conversation is presented as this flat chat prompt that doesn't let me interact with any of the widgets that were previously produced except through more chatting, really makes me feel like the whole product is just doing predatory social-media style "increase time on site" optimization, just trying to lure me into further repetitive and unreliable chats, rather than letting me get in, solve my problem, and get out.

Context Visibility

Managing the LLM context is the ongoing challenge facing organizations that are trying to use "agentic" workflows. Filling up the context with too much information causes well-known problems. In response, advanced LLM users attempting to solve larger problems must break up very long prompts into "skills", give access to lengthy information via "tools", and delegating sub-problems to "sub-agents" rather than simply extending a single prompt indefinitely.

All of these strategies have flaws, because even on the largest models, compared to the breadth and depth of knowledge-work problems, LLM contexts are quite small.

And yet, none of these products will show the context to the user by default. There are third-party addons that can show you a simple progress bar but for addressing the premier engineering difficulty with this technology, that is below the bare minimum.

This lack of visibility means that almost all of the tools for extending the context are flying blind. Rather than responding meaningfully to a full context, everyone just kind of guesses how much state they need by guessing and trying over and over again with progressively more elaborate skill and sub-agent layouts. Even managing context compaction ends up being an advanced API-driven workflow⁵.

A serious product that was trying to help the user understand would not only show "available context" but explain the impact of context compactions, make it easier to see harness-generated prompts, and so on. This would be a first-class feature, combined with the aforementioned reproducibility / replay tools, would allow users to do real experiments to develop an understanding about how to make good use of the context window.

A Sandbox That Actually Works

I've been focused on the chatbot interface here because it is the most immediately egregious upon looking at the UI. But the "agentic loop" tools used for coding are equally dangerous, if not more so. Coding tools keep destroying everyone's data, over the course of years.

These catastrophic incidents that become front-page news are relatively rare compared to the amount of coding-agent use out there. But they also aren't the only kind of sandbox violation. Coding models will so routinely edit test code instead of the system under test that there are "pro tips" articles all over the web giving you the flawed advice to simply ask the agent not to cheat. News write-ups of the catastrophic incidents themselves will also offer glib and wrong advice, like "use a docker container". That might prevent it from literally deleting your operating system, but it won't prevent it from destroying all the local work you have in your codebase (it needs access to a checkout, after all!)

There is a flurry of activity in the infosec space where people are rushing to plug the gaps left by these coding harnesses. Everyone's got their own version of an MCP approval gateway where you can optionally place a proxy between your agent and your production infrastructure.

In the best case, though, all these mitigations and proxies and prompts simply turn the user into an auto-approval automaton, hitting Y, Y, Y, Y over and over again, until you finally are driven mad and hit "yes to all", turn on full-auto mode and submit yourself to the void. With nothing between your personal vigilance and disaster, there are no workflows left beyond decrementing your own vigilance until there's nothing left and then hoping the disaster never arrives.

The fact that some mitigations exist that can be deployed by extra-cautious users does not change the fact that "agentic coding" is an unsafe-by-default technology deployed without concern or guidance. Every frontier lab has tied a spring-loaded shotgun to a dog; the fact that dog owners can publish thoughtful blog posts explaining how you can teach your dog the basics of gun safety or how you can have your dogs play in a bullet-proof room does not mitigate the fact that the product should not have been allowed in the first place, nor should it continue to exist without VERY strong security controls.

I might believe that a frontier lab were seriously interested in providing developers with a useful tool if they shipped something that had safety built-in.

That means tools in the harness, detached from any LLM, independent of the prompt, that could:

• sandbox all filesystem operations and strictly limit ANY deletions outside of specified scopes, regardless of operating system,
• enforce snapshotting of the entire repo on every operation for easy rollbacks and minimal lost work,
• remove the disaster of "auto mode" (not to mention nonsense like --dangerously-skip-permissions) entirely, and
• carefully consider a structure for presenting plans to the user where, rather than provoking immediate alert fatigue by asking for checks on every action, make structured plans which can be submitted to the user as a group of actions and reviewed and approved as a batch.

In the same way that I suggested above that research-based tasks should have a way of re-issuing prompts to determine how reproducible a result is, or whether other sources might be found, agent-based tasks should have a way of being executed against mock services for popular APIs, so that the verification can match both on the front-end (review the plan for making the API calls before they're executed) and the back end (review the API calls that were issued to the mock service and verify that they matched).

Instead, the frontier labs provide us products that are disasters out of the box, give us "best practices" to build massive and elaborate, as well as incomplete and error-prone, security perimeters of our own design. Then they blame "operator error" when it inevitably goes wrong. I cannot believe that these design choices are intended to help us be productive.

Bonus: Human Processes

Organizations deploying AI also frequently come across as unserious, for similar reasons. In 2023, naive exuberance could perhaps be forgiven. But today, as we near the close of 2026, there are several well-known problems, that have been extremely well-covered in the press. None of these things should be surprising, but most orgs deploying these tools are still just letting them rip and hoping it all works out.

Organizations deploying these tools would need at least three kinds of major modifications to their internal processes, if they wanted to be serious about using them safely:

1. Shift Rotations to Prevent Vigilance Decrement

There have been several high-profile incidents where software developers' gradual acquiescence to accepting LLM output have lead to serious economic consequences for the companies deploying them, perhaps best typified by Amazon's "millions of lost orders" due to a gradual decay of their engineering processes from LLM use.

These outages, and other AI-related failures, are due to the difficulty of maintaining focus on the same problems. In other words, as I described above, vigilance decrement is a constant problem, because AI outputs are most often correct, but continue to be incorrect in surprising and non-intuitive ways. As I have previously written, you cannot trust yourself to catch every bug with code review, and LLM output.

Aviation, for example, has very strict rules around rest requirements. There is also a specific rule that "No certificate holder may operate an aircraft without a second in command if that aircraft has a passenger seating configuration, excluding any pilot seat, of ten seats or more.". Other safety-critical professions have similar rules.

And yet, even in the age of the supposed "AI revolution", most software teams are still assigning every engineer a full feature load, not planning for any rest, and telling people to review code whenever they happen to have some "free time".

Maintenance of vigilance has to be your top priority. Regular, scheduled, inviolable rest periods where people do work without AI assistance, and are not exposed to any AI output for review or otherwise, would be crucial in order to stay mentally sharp enough.

The tools themselves should have this sort of thing built in. The mistake-review process described above should have a periodic spot-check mode where a second reviewer periodically reviews a chatbot log, doing their own independent verification of claims, to see if they spot the same errors. This could provide a feedback loop to determine how much rest is necessary to maintain continuous attention and actually spot hallucinations.

2. Skill Practice To Prevent Skill Loss

It is also well-known that AI use leads to AI reliance, and AI reliance leads to skill loss.

I like to use the analogy to dockworkers at a seaport⁶ adopting automation.

If you employ dockworkers to load and unload ships all day long, they are going to be getting tons of exercise. They will be able to lift heavy objects on demand, whenever. They might have plenty of health problems and injuries from this type of work, but "lack of exercise" will not be a problem.

With the development of standardized container ships and mechanized cranes, you are going to be changing their job description substantially: now they mostly spend all day sitting in a small cubicle moving a control lever back and forth, not lifting heavy stuff. They will get worse at lifting heavy objects.

In this analogy however, the cranes are not all that reliable. We know they break, and they drop their payloads sometimes, and the stuff needs to be manually moved. But this only happens a few times a week, at most. If you need whoever is driving the crane to be able to jump out at any moment and still move stuff around manually, then you need to make an affordance for that. You need to give them time to go to the gym and do some lifting for practice, or every crane failure is going to be a major emergency.

An organization doing an AI transformation would also need a massive increase to learning & development budget, both in terms of resources and in terms of schedule. If your people are going to lose skills because they've lost regular practice in the incidental course of doing their duties, then they are going to need deliberate, intentional, non-incidental practice of those skills to keep them sharp.

But rather than trying to accommodate new workflows and give time for people to adjust, most AI mandates are simply dropped on workers like a ton of bricks, with no time to adapt and no affordance for maintaining their skills. Operate the crane and stay fit and healthy and ready to switch back to manual lifting at any time and then get back in the crane cockpit right afterwards. Don't mess up.

Then an accident happens and everyone is surprised, as if this process weren't practically designed to produce a terrible result.

3. Mental Health Resources to Deal with Mental Health Risks

AI psychosis often begins with practical problem-solving, and beyond that, it can start specifically at work. Not to mention the more pedestrian condition of "AI brain fry".

If you are mandating your employees to use a hazardous tool that may seriously and directly damage their mental health, you need trainings and resources. You need in-house therapists and you need to be making sure to check in with people actively to make sure that this is not happening.

Again, the tool itself ought to have some way of dealing with this. An occasional "take a break" popup is easily dismissed; they need a user-visible AI personal dosimeter so you can see your cumulative usage over time.

I don't even know if "usage over time" is a sufficient metric to gauge risk. Maybe if your work chatbot start to talk about resonance too much, unless you literally work as an acoustic engineer, that should be flagged for someone.

We are, again, years into dealing with these tools, and we know these risks exist. Yet no serious mitigations are provided. Not even any way of measuring the risk exposure.

And More

There are also many other risks associated with the technology. There are intellectual property risks with the foundation models, due to recklessness with their training data. There are existential financial risks associated with the infrastructure build-out. The extent to which most "open" models are simply derivatives of frontier models is an open question.

What I Think

If any one of these things were regularly overlooked by AI vendors or users, that would be a totally normal product oversight. Room for improvement for the next version, but nothing catastrophic.

Shipping without any of them doesn't seem like lean product management, it seems like a careless attitude towards risk and a product design philosophy oriented entirely towards short-term demos, with no regard for how to realize actual productivity gains.

Furthermore, being available for years without anything like these features, despite hundreds of incidents demonstrating the risks, with hundreds of billions of dollars of funding, makes it seem to me like if they were to add all the features that would make their product actually safe and hypothetically useful, these features would reveal that it is actually not an improvement to productivity.

In the year since I first wrote about measuring the cost/benefit ratio of AI, I have heard from numerous people who have shown this to management to try to illustrate why their AI initiatives - like almost all AI initiatives - were either failing or burning out their engineers.

I've also heard from lots of people that have told me that it's obviously useful and they don't need to measure so carefully, because they are getting lots of work done that they couldn't have otherwise.⁷

I have yet to hear from a single person who has said "yeah, we measured according to your methodology⁸, and it turns out that our AI work is going great and that our ratio is 0.75".

Obviously, I cannot say for sure why this is; absence of evidence is not evidence of absence. But at this point I think the null hypothesis is that AI tools provide, in aggregate, zero value. They make mistakes too often, and the externalities they produce are so bad and so difficult to control that even before we get to the places where they are just physically poisoning people, even the negative effects on their direct users end up cancelling out whatever benefit to they provide to their organizations.

If I were wrong, then including tools to measure an AI's effectiveness at the tasks their users are actually trying to accomplish, rather than meaningless benchmarks, would show big productivity gains. The frontier labs would be champing at the bit to add such features, and crowing about their fantastic results.

I think the labs know that if they did that, it would present a grim picture to their users. Such tools would let their users see that it's making mistakes much more often than they realized, that they're spending much more time with it than they want to be, and that it's just generally not fit for purpose.

If they prove me wrong by adding in all of these safety mechanisms, and in the process, they make all of their AI technology less harmful, I'll be thrilled to be debunked.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

28 Sep 2026 4:27am GMT