fidzu : python

In introducing wrapture I said the one idea everything sits on is to wrap rather than replace. That is easy to say and harder to see the point of, so this post takes a small piece of code and writes tests for it twice, once with unittest.mock and once with wrapture. I am not going to walk through every mock idiom and show its wrapture spelling, since a good part of the time the two are doing the same thing with different syntax, and the comparison page in the documentation already maps one onto the other. What I want to show is the handful of cases where the difference is structural, where wrapping the real code lets a test say something that substitution cannot.

The code under test

An order service which takes a payment through a gateway, records it in a ledger, and sends a notification. If the ledger write fails, the payment is refunded and the error propagates. The collaborators can be injected through the constructor, so there is a seam for a mock to sit behind, and the payment step goes through a private method on the service itself.

class Gateway:
    def charge(self, amount, currency="USD"):
        return {"id": f"ch_{amount}", "amount": amount}

    def refund(self, charge_id):
        return {"id": f"re_{charge_id}"}


class Ledger:
    def record(self, entry):
        return f"led_{entry['id']}"


class Notifier:
    def send(self, message):
        return True


class OrderService:
    def __init__(self, gateway=None, ledger=None, notifier=None):
        self.gateway = Gateway() if gateway is None else gateway
        self.ledger = Ledger() if ledger is None else ledger
        self.notifier = Notifier() if notifier is None else notifier

    def place(self, amount):
        charge = self._take_payment(amount)
        try:
            self.ledger.record(charge)
        except Exception:
            self.gateway.refund(charge["id"])
            raise
        self.notifier.send(f"order {charge['id']} placed")
        return charge

    def _take_payment(self, amount):
        return self.gateway.charge(amount)

Where the two look the same

Stubbing a return value is the bread and butter of both tools, and on the surface there is nothing to choose between them:

from unittest.mock import patch

def test_stub_with_mock():
    with patch.object(Gateway, "charge", return_value={"id": "stub", "amount": 0}):
        assert OrderService().place(500)["id"] == "stub"

import wrapture

def test_stub_with_wrapture():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub", "amount": 0}):
        assert OrderService().place(500)["id"] == "stub"

Even here there is a difference under the surface. A mock only checks a stubbed call against the real signature if you asked for autospec=True, so without it a call that has drifted from the method's signature returns the stub happily:

def test_drifted_call_with_mock():
    with patch.object(Gateway, "charge", return_value={"id": "stub"}):
        assert Gateway().charge(500, bogus=True) == {"id": "stub"}

That test passes. A wrapture binding is strict by default, so a call the real method would have rejected is rejected by the stub too:

def test_drifted_call_with_wrapture():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub"}):
        with pytest.raises(TypeError):
            Gateway().charge(500, bogus=True)

The error names the site and the problem:

TypeError: orders:Gateway.charge (stubbed): got an unexpected keyword argument 'bogus'

There is a strict=False option for the rare patch which genuinely means to accept a different shape, but the default is the direction I care about. A test which passes because the stub was more forgiving than the real code is a test that will be wrong in production.

Calls an object makes to itself

The mock approach to testing OrderService is to inject a MagicMock as the gateway and assert on what it recorded:

from unittest.mock import MagicMock

def test_self_call_with_mock():
    gateway = MagicMock()
    service = OrderService(gateway=gateway)
    service.place(500)
    gateway.charge.assert_called_once_with(500)

Look at what the mock actually saw, by printing gateway.mock_calls after the call:

[call.charge(500),
 call.charge().__getitem__('id'),
 call.charge().__getitem__().__str__(),
 call.charge().__getitem__('id'),
 call.charge().__getitem__().__str__()]

The charge() call is there, followed by a trail of fabricated chains as the service reached into a return value that was never a real dictionary. What is not there, and cannot be, is _take_payment(). The call from place() to _take_payment() never crosses the seam the mock sits behind, so as far as the test can tell the private method does not exist. If you instead reach for patch.object(OrderService, "_take_payment") to get at it, you have replaced it, and now the real payment logic does not run and the gateway is never charged. Either the method is invisible or it is gone.

With wrapture the binding is on the class, so a call the object makes to itself passes through the wrapper like any other:

def test_self_call_with_wrapture():
    take_payment = wrapture.binding(OrderService, "_take_payment")
    charge = wrapture.binding(Gateway, "charge")

    with wrapture.timeline(take_payment, charge) as tape:
        OrderService().place(500)

        take_payment.events.with_args(amount=500).assert_once()
        assert tape.parent_of(charge.events.first) is take_payment.events.first

The second assertion says the charge happened inside the payment step, and tape.tree() shows the same thing:

orders:OrderService._take_payment(amount=500)  -> {'id': 'ch_500', 'amount': 500}
  orders:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}

Real arguments, normalised against the real signature so the currency default appears even though the caller never passed it, and real return values, nested the way the calls actually nested.

Running the real code while changing one thing

This is where substitution runs out of road entirely. Mock(wraps=real) will forward calls to the real method, but it cannot change the arguments the real method receives, and it cannot touch the result on the way back. The standard library has no way to say "run the real method, but change one thing about it".

For wrapture that is the ordinary case. Here the real charge() runs and only the id in its result is rewritten, which is useful when a real id would be unstable across runs but everything else about the result matters:

def test_pinned_result_with_wrapture():
    charge = wrapture.binding(Gateway, "charge")
    charge.on_call.transforms_result(lambda r: {**r, "id": "ch_TEST"})

    with charge:
        assert OrderService().place(500) == {"id": "ch_TEST", "amount": 500}

transforms_args() does the same on the way in, and validates_args() and validates_result() check without changing. These are stages, and they compose, so a binding can rewrite one argument and check the result at the same time while the real code does the actual work in between.

Asserting on what did not happen

The tests that matter most are usually on the error paths, and the interesting fact on an error path is often an absence. When the ledger write fails, the refund must be issued and the notification must not be sent. Here it is with mock, and it takes three doubles to write:

def test_error_path_with_mock():
    gateway = MagicMock()
    ledger = MagicMock()
    ledger.record.side_effect = OSError("disk full")
    notifier = MagicMock()

    service = OrderService(gateway, ledger, notifier)

    with pytest.raises(OSError):
        service.place(500)

    gateway.refund.assert_called_once_with(gateway.charge.return_value["id"])
    notifier.send.assert_not_called()

The refund assertion is the awkward part. Because the gateway is a mock, the charge id is a fabricated MagicMock rather than "ch_500", so the only way to assert on it is to ask the mock what it invented. The test cannot say "the refund was for the charge that was taken", only "the refund was passed whatever charge() returned", which is the same thing only if you trust the code you are testing. (With a plain Mock rather than MagicMock the test does not even get that far, since charge["id"] fails with a TypeError about subscripting.)

With wrapture the failure is injected at the ledger and nothing else is touched:

def test_error_path_with_wrapture():
    charge = wrapture.binding(Gateway, "charge")
    refund = wrapture.binding(Gateway, "refund")
    record = wrapture.binding(Ledger, "record")
    send = wrapture.binding(Notifier, "send")

    record.on_call.raises(OSError("disk full"))

    with wrapture.timeline(charge, refund, record, send) as tape:
        with pytest.raises(OSError):
            OrderService().place(500)

        refund.events.with_args(charge_id="ch_500").assert_once()
        send.events.assert_never()
        tape.assert_order(charge, record, refund)

The real gateway was charged, so the refund is asserted against the real charge id. The notifier is real and was never called. And assert_order() says the refund came after the failed ledger write, across three different bindings. The tape shows exactly that:

orders:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}
orders:Ledger.record(entry={'id': 'ch_500', 'amount': 500})  !! OSError (injected)
orders:Gateway.refund(charge_id='ch_500')  -> {'id': 're_ch_500'}

When an assertion fails, the message shows what was recorded rather than just the count that was wrong. Asserting on a refund for the wrong id gives:

AssertionError: expected exactly 1 event(s), got 0
<EventLog orders:Gateway.refund[charge_id='ch_999']: 0 event(s)>
    (no events)
  filtered from:
    <EventLog orders:Gateway.refund: 1 event(s)>
        orders:Gateway.refund(charge_id='ch_500')

The "filtered from" section is there because an over-narrowed filter producing an empty log is the easiest way to get a wrong assertion, and showing what the filter discarded is the fastest way to see it.

How the tests are shaped

Beyond what the assertions can say, the shape of the test code changes in a few ways that are worth pointing out.

A binding declares a target without touching it. binding() never patches, so bindings can be created at module scope, given behaviour, and shared, with each test applying and removing them. Nothing happens until apply(), a with block, or a timeline. That separation of declaration from effect is what lets the four bindings in the error path test be declared up front and read like a cast list.

The with block is the primitive, and for a test that binds one or two targets around its whole body there is a decorator form that says the same thing without the nesting. The bindings arrive as keyword arguments, and expectations can be declared on the decorator and are verified when the test finishes:

@wrapture.taped()
@wrapture.bound(Ledger, "record").on_call.raises(OSError("disk full"))
@wrapture.bound(Gateway, "refund").expect_once()
@wrapture.bound(Notifier, "send").expect_never()
def test_error_path_with_decorators(tape, record, refund, send):
    with pytest.raises(OSError):
        OrderService().place(500)

That is the same test as before with the assertions moved to the top as a contract, and a body that only performs the action. An expectation with nothing recording is an error rather than a silent pass.

Fixtures work the way you would expect, and because a fixture yields the binding a test can reconfigure it mid-flight, which is how a test walks a collaborator through failing and then recovering:

@pytest.fixture
def stub_charge():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub", "amount": 0}) as charge:
        yield charge


def test_gateway_recovers(stub_charge):
    stub_charge.on_call.raises(TimeoutError("down"))
    with pytest.raises(TimeoutError):
        OrderService().place(500)

    stub_charge.on_call.returns({"id": "retry", "amount": 0})
    assert OrderService().place(500)["id"] == "retry"

Finally there is an opt-in pytest plugin, enabled with one line in conftest.py, which fails any test that leaves a binding applied and attaches the tape's tree to the failure report of any test that recorded one. The leak sweep is the thing I would turn on first. A patch that leaks changes the behaviour of every test that runs after it, and I suspect plenty of people have lost hours to that without ever knowing which test was the culprit.

Where mock still fits

Everything above follows one rule, which is to wrap the real code and record what actually flowed, with one deliberate opt-out. When the test itself must supply the thing being called, because the code under test receives a callback or a collaborator rather than importing one, stub() supplies a single callable and mock(Spec) a whole object. Both are strict, in that signatures are checked and a name the spec does not have raises, and both record onto the same tape as everything else.

What wrapture does not provide is a spec-less MagicMock() whose attributes exist on first touch and whose call chains all answer. That is the thing which would let a misspelled gateway.chargee.assert_not_called() pass silently in the tests above, and it is left out on purpose. If a test wants an object invented as it is touched, unittest.mock is the tool for that, and it has the other advantage of being in the standard library, on every team's common ground. The two coexist in one suite without difficulty, and the comparison page is there for translating between them.

What's next

The error path test leaned on the timeline and the tape without much explanation of what they are or what an event holds. That is the subject of the next post, where the example is a resource leak, which is a bug nothing in a return value will ever tell you about.

31 Aug 2026 9:38pm GMT

For the best part of two decades, through wrapt, I have been dealing with the mechanics of monkey patching in Python. Anyone who has followed wrapt will know I am quite pedantic about correctness, to the point of caring whether a wrapper preserves every last introspectable detail of the thing it wraps. For much of that time I have wanted the same standard from the tools I use when testing code, and unittest.mock, which does what it does well enough, was never designed to give it. A fabricated Mock answers every method call and verifies nothing. A patched call records a flat list of calls, with no return values and no sense of what was called from what. The calls an object makes to itself are invisible, because the substitute never runs the real code at all. What I want a test to be sure of, that the right calls happened, in the right order, with the real code actually running, sits just outside what substitution can express.

Seeing the real calls as they happen, with the real code still doing the work, was also something I had already spent years on for a quite different purpose. I was the original author of the New Relic Python agent, written while I worked there, and that left me with a lasting interest in instrumenting Python programs. Attaching observation to code you do not control, recording what flows through it, and doing so without disturbing the program being watched, is a problem I have never really stopped thinking about.

Testing and tracing look like different problems, but from where I sat they wanted the same thing, and I had long believed that wrapt's approach of wrapping real code rather than replacing it could serve both. wrapture (the name being wrapt plus capture) is me finally getting around to finding out whether that belief held up.

Wrap anything, capture everything

The one idea everything in wrapture sits on is to wrap rather than replace. A binding names a location in code, a method of a class or a function in a module, and when applied installs a wrapt wrapper around the real callable. Unless you tell it otherwise the wrapper is transparent. The real code runs, with wrapture in a position to watch the call, change it, or answer it instead.

Take a small call graph where an order service charges a payment gateway and then records the result in a ledger:

import wrapture

class Gateway:
    def charge(self, amount, currency="USD"):
        return {"id": f"ch_{amount}", "amount": amount}

class Ledger:
    def record(self, entry):
        return f"led_{entry['id']}"

class OrderService:
    def __init__(self):
        self.gateway = Gateway()
        self.ledger = Ledger()

    def place(self, amount):
        result = self.gateway.charge(amount)
        self.ledger.record(result)
        return result

None of these classes import wrapture or know they are about to be observed. Bindings are created by naming the methods, and a timeline opens a recording scope in which every call through them lands on a tape:

place = wrapture.binding(OrderService, "place")
charge = wrapture.binding(Gateway, "charge")
record = wrapture.binding(Ledger, "record")

with wrapture.timeline(place, charge, record) as tape:
    OrderService().place(500)

print(tape.tree())

Running this the output is:

__main__:OrderService.place(amount=500)  -> {'id': 'ch_500', 'amount': 500}
  __main__:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}
  __main__:Ledger.record(entry={'id': 'ch_500', 'amount': 500})  -> 'led_ch_500'

That is the call graph as it actually ran. The arguments are normalised against the real signatures (so charge(500) and charge(amount=500) look the same), the return values are the real ones, and the nesting comes from what really called what. The tape.tree() call is a convenience for debugging and for demonstrations like this one; in a test you would query the tape instead, and outside of a test the events would be going to a sink, which I will come to.

The same bindings intervene as well as observe. The real method can be stubbed out, made to fail, or left running while one thing about the call is changed on the way in or out:

gateway = Gateway()

with wrapture.binding(Gateway, "charge").on_call.raises(TimeoutError("down")):
    gateway.charge(500)

Inside the block the call raises TimeoutError, and after the block exits the original method is back exactly as it was.

Three uses of one mechanism

That is the whole mechanism. What makes it interesting is that it serves three purposes which are usually handled by three different tools.

The first is plain monkey patching. wrapt's wrap_object() has always been able to patch a target, but it leaves the bookkeeping to you. wrapture adds a lifecycle and a vocabulary over the top of it. A binding declares a target without touching it, apply() installs the patch, remove() restores the original, suspend() makes it inert in place, and a group of bindings applies and removes as one unit. Behaviour is configured on the binding, with returns(), raises(), transforms_args(), transforms_result() and a few others, and can be scripted to change over time, so "succeed twice, then time out" is three lines rather than a hand-written counter. This layer is useful on its own with nothing else switched on.

The second is unit testing, which is where the recording comes in. Because the real code runs, a test can assert on how calls actually flowed through it, and the interesting cases are the error paths. Inject a failure at the gateway, then check that the ledger was never written:

with wrapture.timeline(place, charge, record) as tape:
    charge.on_call.raises(TimeoutError("down"))

    try:
        OrderService().place(500)
    except TimeoutError:
        pass

    record.events.assert_never()

    print(tape.tree())

With the assertion passing, the tree shows where the failure was injected and how it propagated:

__main__:OrderService.place(amount=500)  !! TimeoutError
  __main__:Gateway.charge(amount=500, currency='USD')  !! TimeoutError (injected)

When a test must supply a stand-in, because the code under test receives a collaborator rather than importing it, wrapture provides stub() for a callable and mock(Spec) for a whole object, and these record onto the same tape as everything else. Both are strict: signatures are checked and nothing is invented on first touch. There is deliberately no spec-less Mock() equivalent, and the comparison with unittest.mock in the documentation explains why, alongside a mapping of each mock idiom to its wrapture counterpart. An opt-in pytest plugin sweeps each test for patches left applied and attaches recordings to failure reports.

The third use is ad-hoc tracing of a running application, including one you cannot modify or redeploy. Take the bindings, drop the test around them, and the only remaining question is where the events go. A sink answers that. In practice this is done with a wrapture.toml file naming the targets and the sink, and no code at all. Here is one for a slightly bigger version of the shop above, where the gateway declines some cards and the order service logs a warning when it does:

[[observe]]
target = "shop:OrderService"
name = "place"

[[observe]]
target = "shop:PaymentGateway"
match = "*"
exclude = "_*"

[[observe]]
target = "shop:Ledger"
name = "record"

[[log]]
name = "shop.*"

[[sink]]
type = "printer"

Running the program as python -m wrapture main.py applies the config before the program starts, so the patches are in place before the application imports anything, and the printer sink writes the trace to stderr as it happens:

shop:OrderService.place(order_id='order-1', amount=30, card='5100-0010')
  shop:PaymentGateway.charge(amount=30, card='5100-0010')
  shop:PaymentGateway.charge -> 'ch_30' [7us]
  shop:Ledger.record(order_id='order-1', amount=30)
  shop:Ledger.record -> 'ledger:order-1:30' [5us]
shop:OrderService.place -> 'ch_30' [234us]
shop:OrderService.place(order_id='order-2', amount=240, card='5100-0020')
  shop:PaymentGateway.charge(amount=240, card='5100-0020')
  shop:PaymentGateway.charge -> 'ch_240' [4us]
  shop:Ledger.record(order_id='order-2', amount=240)
  shop:Ledger.record -> 'ledger:order-2:240' [3us]
shop:OrderService.place -> 'ch_240' [105us]
shop:OrderService.place(order_id='order-3', amount=75, card='4000-0030')
  shop:PaymentGateway.charge(amount=75, card='4000-0030')
  shop:PaymentGateway.charge !! PaymentDeclinedError [4us]
  log shop.orders WARNING 'order order-3 declined'
shop:OrderService.place !! PaymentDeclinedError [260us]

Unlike the tidy reconstruction from tape.tree(), this is the live view, with an opening line as each call begins and a closing line with the outcome and how long it took. The [[log]] entry captures the application's ordinary logging calls as events too, so the warning appears nested inside the call that logged it rather than somewhere in a separate log file. With autowrapt installed, even the launcher is unnecessary: AUTOWRAPT_BOOTSTRAP=wrapture in the environment applies the same config at interpreter startup, so the program runs with plain python. That covers the case where something else owns the command line, like a container entry point or a WSGI server.

The printer is the simplest sink. Others stream events to disk as JSON lines, count without retaining, and compose with fan-out, sampling and filtering. Sitting on top of the tracing layer is OpenTelemetry export: with the wrapture[otel] extra installed, one [otel] table in the config sends the same events to any OTLP backend as spans, metrics and correlated logs. Every tree of events carries a W3C trace id, and the id arrives and leaves in traceparent headers, so two services both observed by wrapture join up as one distributed trace without either of them calling an OpenTelemetry API.

The point I want to land is that these are layers of one mechanism and not separate products. The binding vocabulary that stubs a method in a test is the same one that traces it in production, and the config that names methods for a printed call tree is the config that exports spans. What starts as a monkey patch or a test assertion can grow into observability without the code being rewritten along the way.

Where it sits beside what already exists

Part of why I built this is that nothing I could find did all of it. unittest.mock records a flat call list with no nesting and no return values, and a patched call returns a fabricated MagicMock rather than running the real code. Span assertion tools such as OpenTelemetry's InMemorySpanExporter require the code to already be instrumented. Tools built on sys.settrace give you a firehose with no assertion API. APM agents are all-or-nothing products rather than toolkits, and their auto-instrumentation only covers the frameworks they already know about. wrapture needs none of that. You point at your own methods by name and a trace appears, and the same pointing is how it lands in a test, a terminal, or a backend.

Just as important is what it is not. It is not a fabrication tool, and unittest.mock remains the right thing for invented objects. It is not a production APM, although it is a toolkit that APM-like things could be built on. And it is not an OpenTelemetry competitor; it emits to OpenTelemetry rather than trying to replace it.

Pre-built instrumentation

Pointing at your own methods is the core of wrapture, but for common third-party packages the pointing has already been done. The companion wrapture-instrumentation package provides ready-made instrumentation, with Flask and Jinja2 covered so far. Each records a request or a template render as one structured tree, and enabling one is an [[instrument]] entry in wrapture.toml naming the target. Installing the package brings in wrapture and nothing else; the instrumentation for a package you do not have is inert. It is being built target by target, and the instrumentation packages guide describes how to write one for a package not yet covered.

Built with AI, on purpose

Every line of code and documentation in wrapture was written by an AI assistant working under my direction. I want to be upfront about that, and equally upfront about what it was not. This was not vibe coding, where a one-shot prompt produces a pile of generated code and the person driving hopes for the best because they lack the knowledge to judge what came back. Vibe coding has earned its bad reputation. I engineered wrapture carefully from the start. I have spent a long time in this particular corner of Python and knew exactly what the result needed to be, and the AI was the means of producing it rather than the source of the design.

The experiment had two halves. The first was whether an idea I had carried around for years actually held up once built. The second, and just as much the point, was whether a library of this kind could be produced this way, with an AI doing the writing and me doing the directing, to a standard I would be happy to put my name to.

The process is what makes the result worth trusting or not, so it deserves describing. The work started well before any code, with days spent on design documents setting out the goals, the scope, the shape of the API and the layers it would be built in, which the AI and I argued over before implementation began. From there it proceeded in layers, each one specified, discussed, implemented, tested and documented before the next began. Documentation grew with the code rather than after it, and every example in the docs runs as a doctest, so the docs are continually proven against the implementation. Writing them repeatedly exposed designs that read worse than they demoed. The test suite runs against every supported Python version, including the free-threaded builds, on every change. The overhead of Python instrumentation is the usual objection to it, so the recording path was also put through a performance pass, with the cost of a call observed and exported through wrapture measured against the same call instrumented directly with the OpenTelemetry SDK and in the style of its instrumentation packages. The result was comparable per call, with the figures in the OpenTelemetry export guide.

The step I would most recommend to anyone attempting something similar came late. I took the unit test suites of well-known Python packages that lean heavily on unittest.mock and had the AI replicate their tests using wrapture instead, side by side with the originals. Every point of friction became a decision: sometimes a documented position on why wrapture deliberately differs, and sometimes a missing feature that got specified, built and documented like everything else. Several pieces of wrapture exist only because a real test suite could not be expressed cleanly without them.

Throughout, the division of labour was consistent. The AI wrote the code, the tests and the prose. I set the direction, made the design calls, reviewed what came back, and sent plenty of it back. My experience with wrapt and with Python's darker corners is all through the result, in what was asked for as much as in what was refused.

The first commit was in the middle of August and the current release is the eleventh alpha, so this all happened in a bit over two weeks. In that time it accumulated over 1000 tests and over 150 pages of documentation. The documentation is admittedly quite dense in places and needs some work still, but it is complete in the sense that every part of the package is covered. One thing a brand new library has over an old one is coherence. Mature packages accrete features one release at a time, and there is never a moment when the whole API can be redesigned to match what was learned along the way. Because wrapture arrived in a compressed period with the whole design still in view, when validation showed a design could be better it was redesigned rather than worked around.

I know some people are firmly opposed to using AI-written software. If that is you, I understand, and I am not going to argue with your position. It is a reasonable one to hold, and this post exists so you can make the call with the facts in hand rather than discover them later. If AI involvement rules wrapture out for you then wrapture is not for you, and that does not worry me one bit. This has been about finding out whether the process works, and I now have my answer to that. The longer version of all this is on the how wrapture was built page in the documentation.

What's next

wrapture is in alpha, with pre-releases on PyPI. Until 1.0.0 is final a plain pip install wrapture picks up the latest pre-release, so there is no need to pin a version. It requires Python 3.12 or later and wrapt 2.4.0 or later. The API is complete for the three uses described above and I am not expecting it to break, so code written against it today should carry forward to 1.0.0.

What it needs now is use. unittest.mock and OpenTelemetry's own instrumentation are the established tools for the two halves of what wrapture does, and the open question is whether an alternative that does both from one mechanism is something people want. Reports of it working, or not, on real code, and of what confused or was missing, are what will decide whether anything changes before a beta. They go to the issue tracker.

To be clear, wrapture was never premised on anyone else using it. I built it because I wanted to see it exist, not because I had identified a gap in the market. If people find it useful and pick it up, that is great, and I will aim to support it. If there is no interest, I will keep treating it as an experiment and work on it for my own purposes. Either way the questions got answered, and getting them answered was the point.

There is a lot more in wrapture than fits in an introduction, and I expect to write about specific parts of it in follow-up posts, starting with how it can be used for unit testing, and then tracing a Flask application through to an OpenTelemetry backend without touching the application code. For now the getting started page is the place to begin.

31 Aug 2026 9:38pm GMT

In our previous blog post in this series, we discussed state-of-the-art models for object detection: the architectures, the theory, and what makes YOLO12, YOLO26, and RF-DETR tick. If you want the theoretical background on these models, start there.

This post is the practical follow-up: how to actually use these models, how to fine-tune them on diverse, specialized datasets that look nothing like their training data, and how to evaluate the results - all within PyCharm.

Why fine-tune at all?

Every pretrained detector you download was trained on some distribution of images, almost always COCO, which is ~118k training images of everyday scenes containing 80 common object categories (people, cars, dogs, chairs, etc.).

Real-world deployment data rarely looks like COCO. Things that object detection might actually be applied to, such as damaged industrial cables, bone fractures on X-rays, or densely stacked soda bottles on a shelf, are:

• Out of vocabulary: "Bone fracture" is not one of COCO's 80 classes, so the model literally has no output category for it.
• Out of visual distribution: X-ray imagery, industrial close-ups, and heavily occluded shelf scenes differ drastically from consumer photos in texture, viewpoint, and object density.

Deploying a detector on off-distribution data therefore requires fine-tuning. But before we break the models, let's establish that we get similar results on our hardware to the ones reported by developers.

The models

For the purposes of this experiment, we'll focus on three current SOTA object detection families and examine two sizes of each model:

Family	Variants	Implementation
YOLO12	yolov12n, yolov12m	Original authors' repo
YOLO26	yolo26n, yolo26m	Ultralytics PyPI package
RF-DETR	RFDETRNano, RFDETRBase	Roboflow PyPI package

Sanity check: Reproducing COCO val2017 baselines

We're going to be working with six pretrained checkpoints: Two different sizes of each of the three models. To check that these models are behaving as expected, we evaluated all of them on the full 5,000-image COCO validation dataset (val2017) to verify the numbers reported in the previous post:

Model	Params (M)	mAP50	mAP50-95	Latency (ms)
YOLOv12-N	2.55	0.5548	0.4021	23.9
YOLO26-N	2.57	0.5498	0.3952	12.3
YOLOv12-M	19.67	0.6953	0.5259	72.4
YOLO26-M	21.90	0.6906	0.5181	13.9
RF-DETR Nano	30.47	0.6750	0.4835	12.4
RF-DETR Base	32.17	0.7210	0.5325	12.9

Three things stand out even before we leave COCO behind:

1. Larger models (mostly) have better performance. RF-DETR Base leads (0.5325 mAP50-95), but the medium YOLOs get remarkably close (0.5259 / 0.5181) with ~10M fewer parameters.
2. YOLO26's NMS-free design pays off in throughput. YOLO26-N is the fastest model in the lineup (12.3 ms latency) at essentially the same mAP50-95 as YOLOv12-N, which, despite being the smallest model here, is considerably slower (23.9 ms latency). Attention is expensive. (If you want more detail on the model architectures and how they affect performance, see the previous blog post in this series.)
3. RF-DETR Nano is not "nano" by parameter count (~30M - more than YOLO26-M), but it is well-optimized: With 12.4 ms latency, it is the second-fastest overall.

Published papers report optimized inference latency: That is, they measure the model's forward pass in isolation, stripped of the surrounding stages of the object detection pipeline. We deliberately skipped that aggressive optimization so our numbers reflect what you'd actually see when deploying these models.

As a result, our latency figures don't line up with the benchmarks in the models' white papers. There are two main reasons for this:

• Hardware: We used different hardware from the NVIDIA T4 GPU that serves as the de facto standard in object detection benchmarking.
• Unoptimized computation graph: We ran the models in their native framework rather than converting them to TensorRT. TensorRT compiles a network into a hardware-specific engine, fusing layers, selecting the fastest kernels for the target GPU, and optionally running in reduced precision. That can cut latency substantially, but the resulting engine is tied to one GPU and requires an extra build step, so it doesn't represent how these models perform out of the box.

Accuracy is a different story: While our latencies diverge from the published ones, our mAP50-95 results fall within reasonable noise bounds of the reported figures.

Now that we've seen what our pretrained models can do on COCO, the dataset they were trained on, let's see what happens when they're tested off distribution.

The datasets

For evaluation, we used RF100-VL, a large-scale collection of 100 multimodal datasets covering concepts deliberately chosen to be rare in object detection models' pretraining data. These datasets contain exactly the off-distribution targets we care about. These targets also mirror common real-life applications for object detection, giving us a realistic test of these models' capabilities out in the wild.

We picked three datasets that stress test the models in different ways:

Dataset	Domain	Why it's hard	Classes
cable-damage	Technical/industrial	Fine-grained damage types on visually similar backgrounds	break, thunderbolt
bone-fracture	Medical (X-ray)	Entirely different imaging modality; subtle features	angle, fracture, line, messed_up_angle
soda-bottles	Retail	Heavy occlusion, many near-identical instances per image	coca-cola, fanta, sprite

Tutorial: Fine-tuning all three models in PyCharm 🙂

Step 1: Setting up the project

One of the first challenges we had to overcome in this project was that the three implementations do not share a compatible set of dependencies. In particular, the two different generations of YOLO require different versions of the ultralytics package. PyCharm offers a clean solution for this: one PyCharm project with three isolated uv environments - one per model family.

We'll run our computations on a remote GPU. Configuring a remote interpreter in PyCharm follows the same workflow as a local one: the same dialog and the same dropdown as in the local case. Note that remote interpreters require PyCharm Professional; Community Edition supports local environments only.

Firstly, we need to instantiate our three uv virtual environments via:

cd yolov12 && uv venv .venv --python 3.11 

cd yolov26 && uv venv .venv --python 3.11 

cd rf-detr && uv venv .venv --python 3.11

Once your uv virtual environments exist, register each one as an existing interpreter. Go to Settings | Python | Interpreter, click Add Interpreter → Add Local Interpreter, choose Environment as Select existing, and point the interpreter field at that environment's bin/python. PyCharm doesn't create anything here, it just picks up the environment uv already built.

Repeat for each environment. From then on, switching is a matter of picking one from the Settings | Python | Interpreter dropdown, or from the interpreter widget in the bottom-right-hand status bar.

You can find the full list of dependencies required for each model in their respective project repositories. You can either install all the projects' dependencies in PyCharm's built-in Terminal tool window or install individual packages using the Python Packages tool window (including selecting specific versions of packages). You can access both of these tool windows by clicking the relevant icons in the lower left-hand corner of the PyCharm toolbar.

For a step-by-step guide on setting up the environments for all three models, see our GitHub implementation of this tutorial.

Step 2: Getting the datasets

To obtain the out-of-COCO-distribution datasets, we can install our datasets via the rf-detr virtual environment, since it has roboflow as one of its core dependencies. We then set the Roboflow API key as an environment variable so that it is available to the API when downloading the datasets.

pip install roboflow

export ROBOFLOW_API_KEY="your_key_here" # you can get API key here: https://docs.roboflow.com/reference/authentication/authentication/find-your-roboflow-api-key 

After setting everything up, now you can run the Python script below to get the three datasets we're going to use in our tutorial:

import os
from roboflow import Roboflow

api_key = os.environ.get("ROBOFLOW_API_KEY")

if not api_key:
    raise RuntimeError("ROBOFLOW_API_KEY is not set")

DATASETS = [
    "bone-fracture-7fylg",
    "cable-damage",
    "soda-bottles",
]

VERSION = 2          # RF100 projects are generally published at version 2
FORMAT = "yolov8"    # or "coco", "voc", "yolov5"
rf = Roboflow(api_key=api_key)
workspace = rf.workspace("rf100")

for slug in DATASETS:
    print(f"Downloading {slug} ...")

    try:
        project = workspace.project(slug)
        dataset = project.version(VERSION).download(FORMAT)
        print(f"  -> {dataset.location}")

    except Exception as e:
        print(f"  !! failed: {e}")

This script connects to the Roboflow cloud service via its Python API client and downloads three specified RF100 datasets in YOLOv8 format. It loops through each dataset, reports where successful downloads are saved, and prints an error if any download fails.

Step 3: Getting a zero-shot baseline by using pretrained models on custom data

Before fine-tuning, we're going to evaluate the COCO-pretrained checkpoints directly on our three datasets, to see whether the fine-tuning is actually necessary. The result was unambiguous: The models predicted essentially nothing.

Zero-shot mAP50-95 on the test splits of our three datasets:

Model	cable-damage	bone-fracture	soda-bottles
RF-DETR Nano	0.0004	0.0000	0.0027
RF-DETR Base	0.0005	0.0000	0.0004
YOLOv12-N	0.0007	0.0000	0.0266
YOLO26-N	0.0000	0.0000	0.0033
YOLOv12-M	0.0000	0.0000	0.0160
YOLO26-M	0.0000	0.0000	0.0012

This is to be expected; it's not a bug! As the models are closed-vocabulary detectors, that is, they have a finite number of predefined target classes, they physically cannot output a class like fracture that isn't in their 80-class COCO head.

This is the punchline of this whole post: A model scoring 0.72 mAP50 on COCO scores 0.00 on bone fractures. Pretrained ≠ deployable, even when the model is state of the art. Basic machine learning principles still apply, even in the age of AI!

Step 4: Fine-tuning

All models were fine-tuned on a single A100 GPU for 10 epochs. We used standard Ultralytics/RF-DETR fine-tuning pipelines in order to fine-tune the models on our three datasets. We fine-tuned a model for each dataset. The full fine-tuning pipeline can be found in finetune_rf100.py scripts in the project repo, under the folders for each model.

You can see the core of the training setup below. Both YOLO and RF-DETR are built on PyTorch under the hood, but the training loops are abstracted behind higher-level library APIs: Ultralytics' YOLO.train() for the YOLO models, and RF-DETR's own train() functionality.

YOLO12 and YOLO26

train_model = YOLO(args.model)

train_res = train_model.train(
                data=str(yaml_path),
                epochs=args.epochs,
                imgsz=args.imgsz,
                batch=args.batch,
                device=args.device,
                project=args.project,
                name=run_name,
                exist_ok=True,
                verbose=False,
            )

RF-DETR

ModelClass().train(
                dataset_dir=str(coco_dir),
                output_dir=str(output_dir),
                epochs=args.epochs,
                batch_size=args.batch_size,
                grad_accum_steps=args.grad_accum,
                lr=args.lr,
                resolution=resolution,
                early_stopping=True,
                checkpoint_interval=1,
            )

Step 5: Results

Fine-tuning transforms the picture. You can see the results on the test set after training:

On the left, we have the pretrained models' results for the COCO validation dataset. As we showed earlier, accuracy (mAP50-95) fell between 0.39 and 0.53, and all models except for YOLOv12-M showed low latency. The fine-tuned models on the right showed a similar range of accuracy for the cable-damage and soda-bottle detection tasks, only falling lower for the bone-fracture task. Moreover, the fine-tuned models were comparable in latency to the pretrained models for their intended tasks, and for YOLOv12-M, they were even faster. This suggests that, after fine-tuning to the target domain, the models achieve performance that's broadly comparable to the pretrained performance on their original training domain.

Let's now have a closer look at the fine-tuned models' performance, breaking it down by mAP50 and mAP50-95 for the three separate RF-100 datasets:

Model	cable-damage	bone-fracture	soda-bottles
RF-DETR Nano	0.9195 (0.4391)	0.2317 (0.1136)	0.9617 (0.6223)
RF-DETR Base	0.9281 (0.4456)	0.4474 (0.1915)	0.9688 (0.6332)
YOLOv12-N	0.9236 (0.4378)	0.0911 (0.0532)	0.9677 (0.6343)
YOLO26-N	0.8165 (0.3681)	0.0193 (0.0064)	0.9148 (0.5896)
YOLOv12-M	0.8266 (0.3649)	0.1500 (0.0635)	0.9706 (0.6422)
YOLO26-M	0.8707 (0.3896)	0.2194 (0.1038)	0.9596 (0.6304)

What the numbers say:

• The soda-bottles target is the easy win. Every model lands in the 0.91-0.97 mAP50 band. This is likely due to the fact that the domain (consumer products in photos) is visually close to existing classes in COCO, so only the vocabulary was new. Interestingly, the attention model family does great here, with YOLOv12-M taking the top spot (0.6422 mAP50-95).
• cable-damage: Detection is easy, but localization is hard. mAP50 reaches 0.93, but mAP50-95 tops out at 0.446. It appears that the models find the damage reliably, yet they struggle to box thin, elongated defects precisely. If your application needs tight boxes at high IoU, this gap would be a significant issue.
• bone-fracture remains genuinely hard. The best model (RF-DETR Base, 0.447 mAP50) is far from production-ready, and the performance spread across models is huge. The modality shift from photos to X-rays means the pretrained backbone features transfer poorly. The different image modality and small, sometimes almost indistinguishable bone fractures make the detection task way harder than the one employed on common objects identification. This is the dataset that would most benefit from domain-specific pretraining, more data, or longer fine-tuning.
• RF-DETR Base is the most consistent performer, winning on two out of three datasets and challenging seriously for the third. The DETR-style architecture seems to transfer more robustly to unfamiliar domains.

Qualitative results

To visually assess how these models perform, we can overlay the predicted bounding boxes on the images. Let's look at the objects our models detected in six random images per class:

We can see this confirms the accuracy values we saw above: The noisy images of soda bottles in fridges are labeled accurately, with tight bounding boxes for each object. The cable damage is identified less consistently, with some models failing to find the damage altogether, and others creating unnecessarily large bounding boxes. Finally, the images of broken bones contrast sharply with the other two, with less than half of the images having any break identified, and different models identifying different potential breakage points.

Conclusions

Pretrained object detectors are powerful, based on advancements in model architecture over the past five years, but as we've seen here, pretrained does not necessarily mean deployable. All six models performed well on COCO, yet when we applied those same checkpoints directly to our specialized datasets, their performance fell close to zero. However, fine-tuning completely changed that picture.

After only 10 epochs of fine-tuning, all three model families were able to adapt well to both the cable-damage and soda-bottle datasets. As we noted, the soda-bottle task was particularly transferable, likely because it contained objects similar to those contained in COCO. cable-damage was also detected relatively reliably, although the larger gap between mAP50 and mAP50-95 showed that precisely locating these tiny defects was still challenging for all of the models. However, bone-fracture was a completely different story, likely because moving from the sort of natural images contained in COCO to X-rays is a much larger domain shift. While RF-DETR handled this jump best, even its performance shows the limits of fine-tuning, and there are times when you might need to consider more data, longer training, or even domain-specific pretraining.

The broader takeaway is that there is no single "best" detector: It is dependent on the task. Model size, latency requirements, licensing restrictions, and most importantly, the similarity between the model's pretraining data and your target domain all affect the outcome. It is important to refrain from unquestioningly trusting the numbers reported by model providers and explore the fit of a specific model for your own particular task.

Get started with PyCharm today

In this post, we've gone from validating pretrained YOLO12, YOLO26, and RF-DETR checkpoints on COCO to testing them zero-shot on specialized data, to fine-tuning them on three very different object detection tasks, and then finally, comparing the resulting accuracy and latency. Along the way, we've seen how PyCharm can help manage the practical side of a project like this, where multiple model families require different dependency sets and training environments.

PyCharm helps you keep these workflows together in a single project while using isolated Python environments for each model family. Its interpreter management, built-in terminal, Python Packages tool window, and support for remote development make it easier to move between environments and run training on remote GPU hardware without having to manage each part of this workflow separately.

If you'd like to try these experiments yourself, maybe look into fine-tuning these models for your own specific object detection use case! PyCharm is available to download and try. You can use the accompanying project code to reproduce our COCO baselines, download the RF100 datasets, fine-tune the models, and evaluate them using the held-out test splits.

You can find the full code for this project on GitHub. And if you'd like to learn more about object detection, including the architectures behind the models we used in this post, check out the previous post in this series.

31 Aug 2026 1:50pm GMT

---

News

PyCharm & Django Fall Fundraiser

Buy or renew an annual PyCharm Professional license through the campaign link and you get 30% off while JetBrains donates a matching amount to the DSF, which is working to close the gap between the roughly $300,000 it raises each year and the $500,000 that would make a full-time Executive Director sustainable. The campaign runs through September 10, 2026, so buy or renew before then. A renewal adds 12 months to your existing subscription.

---

Django Software Foundation

DEP 0020: Annual Release Cycle

Django moves to one feature release each January under YYYY.N calendar versioning, starting with 2028.0 in place of what would have been 7.0. Every release now gets a year of mainstream support and two years of security fixes, which retires the LTS label.

The Block and Tackle of Django's Code of Conduct Working Group

The machinery behind Django's move to Contributor Covenant 3.0: a 30-day public comment period enforced by a GitHub workflow, CODEOWNERS gating changes to the CoC text, and an automated decision changelog. The working group is releasing its case-tracking templates under CC BY 3.0 for other projects to adopt.

---

Python Software Foundation

RISC-V is now officially supported by CPython!

CPython now supports RISC-V at tier 3, the entry level for new platforms, which means the open instruction set architecture gets ongoing testing on real hardware through buildbots donated by the RISE Project. Stan Ulbrych led the work with Ludovic Henry, Furkan Onder, and Emma Smith, backed by a Sovereign Tech Agency fellowship.

---

Wagtail CMS News

Wagtail 8.0

A read-and-write v3 REST API, custom base page models, a global permission policy registry, and formalized Django 6.1 support. Two smaller wins: StreamField block IDs are now available as template context variables, and AVIF and WebP images are no longer converted to PNG by default.

Wagtail security releases: 7.0.9, 7.3.4, and 7.4.3

7.0.9, 7.3.4, and 7.4.3 carry the same five fixes as 8.0: permission handling in the Pages, Documents, Images, and translation APIs, document identification by SHA1, and snippet copying. Upgrade to the patch release matching your version.

Streamlining content ops with LLMs: Wagtail user guide

Google Summer of Code contributor Raghad Dahi rebuilt the user guide site, retiring a versioning scheme that made editors duplicate the whole site per release in favor of blocks readers can filter by version. For translations, an evaluation suite scored LLM providers on cost and quality before settling on DeepSeek V4 Flash, now covering 52 languages plus right-to-left support.

CMS with AI, not AI CMS: Wagtail 8.0's new API

The thinking behind that API: rather than bolting AI buttons onto the admin, Wagtail exposes 50+ admin operations with OpenAPI docs and Markdown rich text, drivable from curl, a script, or an MCP implementation. Worked examples include fixing SEO descriptions with an LLM and a generic content importer in about 100 lines.

---

Django Fellow Reports

Django Fellow Report - Jacob

Jacob Tyler Walls filed an early report before heading to DjangoCon US, with his usual prolificacy triaging three tickets, reviewing nine, and authoring seven. In addition, engaged with regular security reports and provided 1-1 mentoring to his GSoC mentee, Pravin.

Editors Note

All 3 Fellows are at DjangoCon US this week, so no formal report from Sarah or Natalia. All 3 also gave excellent talks that we will link to when the videos are available later this year.

---

Articles

Fuzzy String Matching in Django and PostgreSQL

Four ways to match misspelled and variant names, with the trade-offs spelled out. It expands on the author's DjangoCon US 2026 talk on search-as-you-type across 54 million names.

Modern Django Deployments in 2026: My DjangoCon US 2026 Conference Talk

The slides and notes from Will Vincent's recent talk on deployments.

Nifty Django Feature: Third-Party Packages

A helpful overview of Django's third-party package ecosystem, where to look, and how to apply it in your Django projects.

When Python is Too Slow

An opinionated guide on where to turn when Python feels too slow in your application (the answer isn't always switch to Rust).

The Move to Python 3 Begins!

CCP is moving EVE Online's 2.4 million lines of Python off Stackless Python 2.7, in place since 2010, with the first changes deployed on August 25. Stage one leans on automated tools to make the code compile under both versions: 95.9% of roughly 20,000 files already do, leaving about 3,300 blocking lines (1,500 print statements, 800 long literals like 123L, 600 old-style except clauses) and another 20,000 lines that compile but behave differently and need a human to look at each one.

Core Dispatch #10

A roundup of CPython development from August 5 to 27: Python 3.12.14, 3.11.16, and 3.10.21 shipped on the 12th, 3.15.0 release candidate 2 is due September 1, and six PEPs moved, including PEP 805 on safe parallel execution and PEP 833 reaching Final for the simple repository API. Most of the discussion energy went to the competing module export proposals in PEPs 842, 843, and 844.

---

Events

DjangoCon Europe 2027 in Austria!

Five days of Django, Python, and community in Innsbruck, Austria, February 17-21, 2027.

DjangoCon US 2027 in Riverside, California

Join us for five days of inspiration, education, and networking at the Riverside Convention Center in beautiful Riverside, California, September 13-17, 2027.

Call for Organizers: DjangoCon US 2027

That Riverside conference needs people to run it, and more than 15 committees are recruiting, from program and sponsorship to Code of Conduct, A/V, website, and sprints. Leadership roles carry the heaviest load, with weekly check-ins and monthly board reports, but most positions do not require previous organizing experience: email hello@djangocon.us to volunteer.

---

Django Forum

Public thanks to our 3 Fellows

A short forum thread appreciating Django's Fellows.

---

Podcasts

Django Chat #205: Django Developers Survey 2026

A special summer episode on the just-released 2026 Django Developers Survey, working through what it says about Django 6.1, HTMX, async, AI, deployment, testing, and Python tooling.

---

Django Job Board

Two foundation roles anchor the board this week, with the DSF hiring its first Executive Director and the PSF looking for a Security Developer, alongside two full stack engineering openings.

Full Stack Software Engineer (Hybrid) at Provision

Executive Director at Django Software Foundation

AI-Assisted Software Engineer, Web Applications at Logical Media Group

Security Developer at Python Software Foundation

---

Projects

15r10nk/matchify

Converts eligible if/elif/else chains into Python 3.10+ match statements, preserving runtime behavior and source formatting, including isinstance checks that become class patterns with attributes.

matiasb/django-tasks-fennel

A Django Tasks backend which uses Celery as its underlying queue. Mentioned as part of the author's talk at DjangoCon US this week: Teach Django Tasks to speak Celery: Building a Celery backend for Django Tasks.

28 Aug 2026 3:00pm GMT

It's been another month of slow writing and not for want, but in between holidays and the same amount of work, the blog post just got squeezed out each week. Additionally the GSoC project that I have been mentoring took a slight turn into more of a research based thing rather than trying to push anything into Django right now.

However with the GSoC research itself and the recent features of tasks, the email updates, my own prodserver package, I'm beginning to solidify in my head what a modern Django feature looks like. One thing to clarify here is when I use the word feature, I'm referencing a concept that Django can represent, such as Databases, Tasks, Emails & Storage.

These features while different in what they achieve have a very similar architecture within Django, marked by some common characteristics:

• A common API for the rest of Django to use
• A single settings configuration, typically a dictionary
• Pluggable backends that do the actual implementation and specified in the above settings
• Minimal backend implementations inside Django (except Databases), with extra implementations provided as community packages

This architecture leans into Django being an API layer for various concepts that all tie together to become a website or web app. I'm taking this route with prodserver and this approach also taken by by mentee for GSoC when producing django-experimental and django-featurevault.

Django Experimental is a proof of concept around how an experimental features may be added to Django at some point. Django Feature Vault is a similar package specifically targeting an API for feature flags native to Django. Please do give them a read and a spin on a project if you like and raise issues. Both also have associated draft DEPs (Experimental, Feature Flags) available to review and comment.

I wonder if we can more formally codify this architecture (perhaps a copier package template?) to continue to smooth the on-ramp for those that want to contribute new features and ideas to Django and the community.

27 Aug 2026 5:00am GMT

A written guide to my talk on deploying Django and why 90% of it is the same.

26 Aug 2026 11:57am GMT

Starting with 0.3.0, Astral's uv brought many great features, including support for cross-platform lock files uv.lock. Together with subsequent fixes, it has become Python's finest workflow tool for my (non-scientific) use cases. Here's how I build production-ready containers, as fast as possible.

06 Aug 2026 12:00am GMT

As I have discussed in previous posts, "AIs" can make mistakes. In fact, they do make mistakes, and their mistake-making patterns are such that where and how they will make mistakes is both uncertain and constantly changing.

Thus, in any scenario where you want to attempt to make "productive" use of "AI", you must have a system in place for checking every result. Not checking some results; checking every result. If each result might have a consequence for you (and if it didn't have a consequence, why bother automating it?) and you cannot predict in advance which kinds of results will need verification, then verification is always required.

The verification often ends up being just as expensive as doing the work in the first place, which means that if you want your usage of "AI" to be personally profitable, you have to find someone else to externalize the cost of verification onto. This person becomes your adversary, and, if you are successful, your "AI's" victim.

The Ladder-Climber And Their Reverse-Centaur Rungs

One way that this constellation of facts can straightforwardly assemble themselves into a dystopian nightmare is the phenomenon, described by Cory Doctorow, of the reverse centaur. This is when your employer non-consensually turns you into the verification system. The "AI" does the fun part of initially performing the work, and then you do the boring part where you check if the robot is right and clean up its messes, even if everyone already knows that it would, in aggregate, be cheaper for you to do the work in the first place.

Reverse centaurs can be made from any automation, not only "AI" automation. I think that there is a reason that this term happens to have emerged in the "age of AI", though, and not with earlier automation technologies (even those which were considerably more viscerally horrific). That reason is: the wrongness of "AI" output is not merely a technical feature that must be compensated for, it is a generalized externality.

As I mentioned above, if you are responsible for the entirety of the work, both extruding the "AI" output and checking it, it's usually cheaper to have humans do the entirety of the work to begin with. When humans do the writing directly, we can check as we go, and thus verification doesn't need to be as comprehensive.

When "AI" coding advocates say "code review is the bottleneck", what they are observing is that the LLM is still rolling the dice for each PR, and a human is still necessary to verify that each of those rolls is a winner. But calling this process "code review" is a bit of a misnomer; it's not really "code review" in the traditional sense, it's human understanding.

Before the advent of "AI", the human understanding was implicit in the process of writing the code in the first place¹, and the code review was a way of diffusing and extending that understanding. Now that the code can be authored with no initial understanding taking place, that cost has not gone away, it has moved.

Human understanding was always the bottleneck.

However, this is taking a collaborative view of a software project, where satisfying the needs and solving the problems of your customers are the goals. We can see that "AI" is a bad tool to satisfy those goals, because all it's doing is converting the first half of the work, that of understanding the code as you write it, to understanding the agent's output as you read it.

What if, instead, we were to take the view that every software company is a Hobbesian nightmare, red in tooth and claw? In this view, the only goal of a software project is for the individual developers to make their promo cycles and get their bonuses. Given that there is only a certain amount of money to go around, this is a zero-sum game where each programmer wants to look more productive than their colleagues.

Pretty much every organization finds it easy to reward "productivity" as expressed by lines of code emitted, but the benefits of doing thorough and thoughtful design, analysis, and code review very difficult to reward. In this world, an LLM is an invaluable tool for the sociopathic ladder-climber, particularly if your legacy organization is still structuring their workflows as if the person prompting the bot is "writing" the code, and then they get to foist off the act of "reviewing" the code onto someone else.

Here, the prompter effectively externalizes the cost of the LLM's failures but internalizes any benefits. The prompter will vibe-code a big feature, so large that the assigned reviewer can't possibly comprehend it all effectively. When this happens, the reviewer will, eventually, be pressured to approve it, even if they can try to spot a few problems along the way. The reviewer has their own work to get back to, after all, the obligation to review the prompter's (read: the bot's) code is a drain on their time that they are not going to get rewarded for.

If this feature is a big success, the prompter gets a promotion. If it causes a big issue, well, the reviewer must not have been careful enough.

This is why LLMs are "good for coding", and also why their biggest promoters keep having outages.

The Generative Gish Galloper

Coding is the biggest "success story" of this type of adversarial communication, but it is by far not the only instance of such a thing. LLMs create a new form of leverage that can turn Brandolini's law from a linear advantage into an exponential one. If you are engaged in a political debate where you want to overwhelm the other side in nonsense, an LLM can generate bullshit faster than it is physically possible for a human being to type, let alone respond thoughtfully. There is an asymmetry to the utility of this weapon as well: only one side of the political spectrum wants to flood the zone and destroy trust in institutions and the concept of truth. There's a good reason that the fascists love it.

Straightforward Spam and Fraud

This is kind of obvious, but LLMs can generate lightly-customized, plausible-looking text much more quickly than any human being. This facilitates their use in fraud, spam, and scams. In a spamming or fraudulent interaction, once again, the costs are externalized onto the victim: the recipient of a spam message has to do all the work of "checking" the LLM's output. Spammers already expect very low hit rates from boilerplate, and if the LLM can increase those percentages from 1% to 5% the technology will pay for itself; they don't need anything like reliable accuracy.

Customer "Support"

If you have any kind of commercial relationship with a company, I probably don't even need to mention this: customer "support" bots are a misery. Everybody knows it at this point. But customer support is usually conceptualized by businesses as an adversarial interaction, because it is a cost center. They maintain internal metrics on time-to-resolution and try to optimize them. Implicitly, this creates a dynamic where the goal of the customer service agent's job is not to solve your problem, but to emit noise that will cause you to think your problem is resolved, or to give up, as fast as possible. Unsurprisingly, LLMs can emit this noise faster than humans can, getting those customers off the phone. But those customers will remember those interactions, and the story outside the TTR metrics is horrible.

Similarly to the situation in software development, LLMs can look very good on paper for customer support, but mostly what they are doing is illuminating the problems with the industry's existing metrics, by turning "winning the metrics battle against the customer" into a more obvious and immediate defeat for the company's long term reputation.

"Education"

In 2026 it is sadly a fact of life that students cheat all the time using "AI", and that this cheating is very successful, in that the teachers find it very hard to detect.

LLMs are great for cheating on schoolwork because the student is externalizing the work of the checking onto the teachers, who are often starting at a disadvantage to begin with, at least in the US.

My view is that this is happening because of a divergence in the way that students vs. teachers (or, more accurately, "the broader educational system") view grading.

When a student is asked to write an essay, the teachers see the effort as both intrinsically worthwhile for the student, as well as useful as a pedagogical tool to evaluate and react to the student's progress. The student, by contrast, sees a stumbling block designed to knock them off the path to success and into a permanent underclass. It is no wonder that the student sees "AI" as useful to their own goals and has no compunction about deploying it.

There is a bitter irony that the ability to understand the inherent value of actually writing the essay on their own is the sort of thing that students can really only learn by writing a bunch of essays. There's no way that I can think of which makes the benefit legible as long as a shortcut is available.

The net effect here is a downward spiral, where the already-wobbling educational system is sustaining an attack that it doesn't have the resources to recover from. The individual students' attacks against their teachers and their schools' grading systems might appear to momentarily succeed, but they will win the battle and lose the war.

Spamming "For Good"?

Usually when we talk about someone unilaterally choosing to enter into an adversarial relationship, that's an "attack" and for good reasons we have a negative impression of the attacker. However, I would be remiss if I did not point out that there are some cases where the relationship was already adversarial; just because you're the attacker doesn't mean that you are evil.

For example we might imagine use-cases like automatically filing appeals for prior authorizations against health insurance. It's relatively well-known at this point that the main way for-profit insurers maintain their margins is by denying claims right up to the line of the policies themselves being fraud, so using a spamming tool to fight them might be entirely justifiable² in that case.

Similarly, using an LLM could be justified in a fight against a company refusing to honor a warranty. One could imagine using an LLM to immediately generate replies and escalations.

However, even in imagined cases like these, the underlying problem is that the insurers and the vendors already have a tremendous amount of structural power, so it is more likely that they will have the advantage in deploying a communications weapon like an LLM, as well as enacting policies to simply ignore any LLM-based communication that you might submit. Worse, if these strategies were to become widespread, they might provide an excuse to reject any communications by feeding them into an unreliable "LLM detector" and issuing an automated "computer says no" even to hand-written correspondence.

It is also worth stressing that these cases are imagined, as compared to the very real coworker-abuse, spam, scam, fraud, and disinformation campaigns being waged in real life today.

Therefore, while legitimate uses might exist, it's hard to imagine that there's anywhere they would be genuinely valuable and sustainable. In the best case "AI" will provide a temporary advantage for underdogs that will provoke an arms race which the resource-advantaged adversaries will win in the long run, in the worst case the arms race itself will cement permanent structural change that will make things worse.

"Search" By Stealing

Most of the adversarial utility of "AI" is on the "write" side, since write-amplification is more obviously aggressive than reading. But the "read" side of LLMs - summarization and question-answering - can be a form of attack as well.

To begin with, the act of reading itself is currently enormously destructive, but that's arguably not a fundamental aspect of this technology. They could set reasonable rate-limits and respect things like robots.txt, as search engines have for decades now. They could also refrain from committing criminal levels of copyright infringement. But, today, using "AI" tools does suborn this sort of out-of-control crawling.

More insidiously, consider the scenario described in this YouTube video. The LTT Bros decided to try Linux again, and in the course of so doing, they had problems. When trying to solve these problems, they were faced with a choice: they could consult Reddit, or they could ask an LLM. Asking an LLM would "gaslight the heck out of" them, but they still found it preferable, because they would at least get an answer without getting yelled at.

Initially this sounds great. But it also means that you want to extract knowledge from a community, while mechanically eliding any values or norms that the community may want to impart as part of offering that knowledge. As someone who spent many years in a community tech support role, this is worrying. Many requests for support are people asking how to do things that will momentarily solve a superficial problem but create a long-term reliability problem or even an immediate security risk, that the question-asker doesn't want to hear about. Consider the question "I'm tired of entering my password so much, how do I make it so my laptop unlocks automatically". An obsequious chatbot will helpfully tell you how to do this without pushback.

But, this is also a sort of ethically murky area. The Linux community is somewhat famously, for many years now, a toxic cesspool of general hostility, misogyny, etc. It is certainly a good thing that people can get access to this knowledge without subjecting themselves to abuse. But it also means that the people with the power and the privilege to change the community for the better can just quietly withdraw, rather than fixing the problems. It also means that the positive elements of culture cannot be transmitted, and people will have no opportunity to learn about unknown unknowns.

In this case, the "adversarial" communication is with society. The thing that using an LLM for search lets you do is withdraw from society and avoid forming any personal connections. There are some personal connections which are painful and annoying, and so that can feel like a momentary balm. But the need to make connections in general is, like, the concept of society itself.

Who Am I Hurting?

LLMs are good at adversarial communication. They are so good at it, relative to their other benefits, that they will tend to make communications adversarial if you are not remaining vigilant about the possibility that it might do so. My request to you, dear reader, if you are going to use such tools, is to always ask yourself, "who might I be hurting, if I use an LLM for this?"

If you're using an "AI", who is its adversary? If you haven't given it one yet, who might the "AI" turn into an adversary? Who might you overwhelm with an asymmetric amount of output, or, if you're receiving information and not sending it, who are you taking that information from without consulting?

Figure out the answers to these questions and conduct yourself accordingly; the answer might be "yourself".

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

23 Jun 2026 8:06pm GMT

Codecov's unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn't rely on third-party services.

09 Jun 2026 12:00am GMT