I wanted to migrate one of my Python packages from mypy to ty and pyrefly. I handed this task over to Claude, and at the end I asked it to write out some guidance on how to perform it most efficiently. So what follows is AI-generated 'slop'.

This guide is not about using fastkml. It documents how fastkml itself was migrated from mypy to Astral's ty and Meta's pyrefly, so the same playbook can be replayed on other codebases with less trial and error. Keep it here because the next migration (human- or agent-driven) should start from findings, not from zero.

Running two checkers instead of one is deliberate, not incidental. ty and pyrefly disagree with each other and with mypy often enough that running only one gives a false sense of completeness. Budget for both, and expect them to catch different subsets of the same bugs.

TL;DR

1. Get a raw error-count baseline for both tools before touching any source. Categorize by error kind, not by file.
2. Look for one systemic root cause before fixing anything file-by-file. In most codebases with an optional C-extension backend (lxml, pydantic-core, orjson, etc.) there is a single architectural fix that collapses 60-90% of the noise.
3. Fix genuine bugs the tools surface (there will be some - both tools are pickier than mypy about Optional/union narrowing, positional-only stubs, and unpacking).
4. Bulk-suppress test-file "constructed-then-accessed-without-narrowing" noise scoped to tests/**, not case by case, and not by widening the rule to error kinds that could hide real bugs (invalid-argument-type is not unresolved-attribute).
5. Turn on strict presets, then promote specific rules the preset doesn't cover, and explicitly cut the ones that create disproportionate mechanical churn (ask a human before doing a 80-site @override sweep).
6. Verify: both tools clean, full test suite green (with and without optional runtime deps installed), linter clean, and the TOML re-parses.

Phase 0 - Inventory the mypy config honestly

Before deleting [tool.mypy], read what each flag actually bought you, because that's the strictness bar ty/pyrefly need to match or exceed:

Also inventory stale per-module overrides - a mypy config that's been edited over years accumulates dead entries (a module path that was renamed or deleted, but the override survived). Grep for the referenced paths; don't carry dead config forward.

Phase 1 - Install both tools and get a baseline

uv pip install ty pyrefly
ty check <src> <tests>
pyrefly check <src> <tests>

Immediately categorize, don't read line by line yet:

ty check <src> <tests> 2>&1 | grep -oE '^error\[[a-zA-Z-]+\]' | sort | uniq -c | sort -rn
ty check <src> <tests> 2>&1 | grep -E '^\s+-->' | sed -E 's/^\s+--> //; s/:[0-9]+:[0-9]+$//' | sort | uniq -c | sort -rn

The second command (errors by file) usually reveals the systemic cause immediately: a handful of files concentrate a disproportionate share of the errors, and they're usually the ones touching an optional/duck-typed dependency.

If a partial migration already exists in the repo (CI switched over but pyproject.toml grew broad ignore-missing-imports = ["*"] / blanket missing-attribute = "ignore" sub-configs), treat that as a red flag, not a starting point. Broad suppressions accumulated during an in-progress migration usually mean someone hit friction and silenced it rather than fixed it. Re-run with those suppressions removed to see the real baseline before deciding what's worth keeping.

Phase 2 - Find the systemic root cause first

The highest-leverage move in this kind of migration is almost never "fix errors file by file." It's finding the one architectural mismatch that both checkers are tripping over identically across dozens of call sites.

The recurring pattern: a project supports an optional, richer backend (lxml over xml.etree.ElementTree, orjson over json, ujson, a C-accelerated regex engine, etc.) via a runtime try/except import, and defines either a Protocol or just relies on structural duck-typing to abstract over both. mypy tolerated this for years via ignore_missing_imports = true, which silently treats the untyped backend as Any everywhere. Neither ty nor pyrefly degrade that gracefully by default - they'll either partially resolve the untyped backend's real (but incomplete) info, or fall back to typing it against whichever branch of the try/except does have full stubs (usually the stdlib fallback), and then report every method/kwarg the richer backend uniquely offers as invalid.

The fix, applied at the import site (not scattered across every call site):

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # Type-checkers see the richer backend's own stubs; the preferred
    # backend's API is treated as a superset of the fallback's.
    from lxml import etree
else:
    try:
        from lxml import etree
    except ImportError:
        import xml.etree.ElementTree as etree

If the project also defines its own Protocol to abstract over both backends (e.g. types.py: class Element(Protocol): ...), consider going one step further and making that Protocol literally alias the richer backend's real type under TYPE_CHECKING, falling back to the structural Protocol only for runtime/non-typechecking purposes:

if TYPE_CHECKING:
    from lxml.etree import _Element as Element
else:
    class Element(Protocol):
        ...  # the original structural protocol, unchanged

This one change collapsed roughly 150 of ~240 diagnostics in the fastkml migration, because it fixed both the "backend-specific kwarg doesn't exist" class of errors and the "structural Protocol isn't assignable to a concrete stdlib parameter type" class in one shot (see pitfall below).

If a stub package exists for the richer backend (lxml-stubs, types-ujson, etc.), add it to your typing dev-dependencies - but read the pitfalls section before assuming it's a strict improvement for both checkers.

Pitfalls (the part worth re-reading before your second migration)

1. # type: ignore[code] is not portable

Neither ty nor pyrefly parses mypy's bracketed error-code suppression the way mypy does.

• ty honors a bare # type: ignore (no brackets) as a blanket suppression for that line, but a bracketed # type: ignore[some-code] is not recognized as a ty-ignore at all - it's inert noise as far as ty is concerned, and the underlying error still fires.
• pyrefly does honor # type: ignore[...] by default (its --enabled-ignores defaults to type,pyrefly), so bracketed mypy comments mostly still work for pyrefly specifically.
• Both tools have their own dedicated syntax: # ty: ignore[rule-name] and # pyrefly: ignore / # pyrefly: ignore[error-kind].

Verify empirically before trusting any of this - behavior can change between tool versions:

def f() -> int:
    return "y"  # type: ignore[return-value]

Run ty check and pyrefly check against a two-line repro before deciding on a suppression strategy for the whole codebase.

Practical rule that worked well: keep the original # type: ignore[code] comment (documents intent, keeps pyrefly happy) and append # ty: ignore[rule-name] on the same physical line for ty. Don't strip the mypy-era comments outright; they're free documentation of why a line is exceptional.

2. pyrefly's TOML keys are snake_case even though its CLI flags are kebab-case

This is the single most time-consuming mistake to make. pyrefly check --replace-imports-with-any 'lxml.*' works from the CLI. Writing the "obvious" TOML equivalent:

[tool.pyrefly]
replace-imports-with-any = ["lxml.*"]   # WRONG - silently different key

...does not raise an error from pyrefly check in some code paths, but it does hard-fail with pyrefly dump-config (unknown variant 'replace-imports-with-any'... Fatal configuration error), and depending on invocation order this can also break pyrefly check itself later. The correct TOML key uses underscores:

[tool.pyrefly]
replace_imports_with_any = ["lxml.*"]

Meanwhile, error-kind names (used as dict keys under [tool.pyrefly.errors] or inside a rules = {...} table) do use hyphens (missing-override-decorator, redundant-cast, etc.) - matching the CLI's --error/--ignore rule-name spelling, not the config-field spelling. There is no single consistent casing convention across the whole config surface; check pyrefly dump-config after every config change, not just pyrefly check, because check can look clean while a nearby key is silently ignored.

After any pyrefly config edit, run both pyrefly dump-config (schema/parse validation) and pyrefly check (behavioral validation) - one catches structural mistakes the other doesn't surface.

3. pyrefly's [[tool.pyrefly.sub-config]] array-of-tables is fragile against interleaving

Pyrefly's per-path overrides use TOML's array-of-tables syntax:

[[tool.pyrefly.sub-config]]
matches = "tests/**/*"

[tool.pyrefly.sub-config.errors]
missing-attribute = "ignore"

TOML allows other, unrelated top-level tables to appear between [[tool.pyrefly.sub-config]] and its paired [tool.pyrefly.sub-config.errors] - the nested table still binds to the most-recently-opened array element regardless of what's interleaved. That means a pyproject.toml that grew organically (auto-migration tooling appending blocks near whatever happened to be at the end of the file) can end up with three sub-config blocks scattered across 100+ lines of unrelated project/tool config, and it will still parse. It becomes a landmine the moment someone (or an agent) deletes one [[tool.pyrefly.sub-config]] header without also deleting its now-orphaned [tool.pyrefly.sub-config.errors] block - the orphaned errors table then either binds to the wrong array element or breaks parsing entirely.

Fix: keep every pyrefly (and ty) config block contiguous in one place in the file, even if that means moving it away from wherever an automated tool first inserted it. Re-parse after every edit:

python3 -c "import tomllib; tomllib.load(open('pyproject.toml','rb')); print('OK')"

4. A Protocol is not assignable to a concrete class parameter

If your duck-typing abstraction is a Protocol (say, types.Element) and internal code passes Element-typed values into functions that are typed against the concrete stdlib/third-party class (xml.etree.ElementTree.SubElement(parent: Element[Any], ...)), both ty and pyrefly will reject it - even though the Protocol is structurally compatible at every call site. Protocol → concrete-class assignability doesn't work the way concrete → Protocol does, and a mutable/invariant attribute (text: str on the Protocol vs. text: str | None on the real class) makes it worse.

This resolves itself for free once you apply the Phase 2 fix (alias the Protocol to the real backend's type under TYPE_CHECKING) for internal, backend-facing code. Keep the original Protocol only for the codebase's genuinely-public, backend-agnostic API surface.

5. Registry/callback-style dispatch can't be narrowed at the signature level

A common pattern: a generic dispatch table stores classes: tuple[type[object], ...] and a matching Protocol requires every registered callback to accept exactly that (necessarily wide) signature, even though any individual callback only ever receives one concrete class at runtime. You cannot narrow an individual callback's parameter type to the concrete class it actually expects (tuple[type[SpecificClass], ...]) - that breaks structural assignability against the wider Protocol the dispatcher requires (parameter types are contravariant; a callback that only accepts a narrower type can't stand in for one the dispatcher will call with the wider type).

Fix at the call site inside the function body instead of the signature: cast("tuple[type[SpecificClass], ...]", classes), or cls = cast("type[SpecificClass]", classes[0]). Keep the public signature honestly wide.

6. **heterogeneous_dict splats can't be validated against multi-parameter constructors

fields = {"type_": DataType.int_, "name": "Integer"}
SimpleField(**fields)

Both checkers infer fields: dict[str, DataType | str] and then check every keyword argument against that whole union, rather than against each parameter's own specific type - because plain dict[str, ...] splatting isn't a TypedDict, so there's no per-key type information available. This isn't a narrowing bug or an inherent-bad-input case; it's a structural limitation of **dict splatting itself. Either convert the dict to a TypedDict (real fix, more invasive) or accept a targeted ignore comment - don't spend time trying to "fix" it any other way.

7. Community stub packages can behave differently per checker

If you add a community-maintained stub package (lxml-stubs, etc.) rather than relying on inline py.typed types, expect it to have its own bugs, and expect those bugs to manifest differently per checker. In this migration, lxml-stubs declares several attributes using the legacy stub syntax tag = ... # type: str (pre-PEP 526). ty tolerates this by falling back to Unknown for that attribute (safe, just loses precision). pyrefly mis-parses it as the attribute's type being the literal value Ellipsis, then reports every subsequent use (.strip(), .split(), slicing) as an error on a nonexistent EllipsisType method - a wave of dozens of false positives that has nothing to do with your code.

There is no fix on your side other than working around the stub bug per-checker:

[tool.pyrefly]
# Force pyrefly to treat the whole (broken-for-pyrefly) stub package as Any,
# while `ty` still gets full value from the same installed stub package.
replace_imports_with_any = ["lxml.*"]

Don't assume "we added the stub package" is the end of the story - verify both tools independently after adding any third-party stub dependency, because "more type information" is not always strictly better across tools.

8. Fixing the root cause makes old workaround cast()s redundant - clean them up

Once you fix the systemic issue, re-run both tools and look specifically for redundant-cast warnings. Every cast("Element", root) that existed purely to placate mypy's ignore_missing_imports fallback becomes genuinely unnecessary once the real type flows through correctly, and leaving it in is now dead weight (and a ty/pyrefly warning) rather than a workaround. This is a good automatic signal that the root-cause fix actually landed.

9. Some "strict" rules are mechanical churn, not bug-catching - decide explicitly, don't default to on

Pyrefly's strict preset (and to a lesser extent ty's optional rules) includes checks like missing-override-decorator (PEP 698 @override), which can flag dozens to low-hundreds of ordinary __repr__/__init__/method overrides in any codebase with meaningful inheritance. This is a legitimate check with real value in large team codebases (catches silently-broken overrides after a base-class rename), but adding @override everywhere is a large, purely mechanical diff disconnected from the actual "fix type errors" task.

Don't silently turn this on or off. Surface it explicitly (to a human reviewer, or via an explicit question if you're an agent) before deciding: disable it with a documented reason, or actually do the sweep. Either is defensible; picking silently isn't.

10. Test-file "narrowing noise" is real but should not become a license to blanket-suppress everything

The overwhelming majority of test-file errors in a mature test suite will be the same shape: construct an object, then immediately access/assign a field typed X | None or A | B | None, without a narrowing assert x is not None - because the test knows the value is set (it just set it three lines up) but the checker doesn't. This is legitimately safe to bulk-suppress scoped to the test tree:

[[tool.pyrefly.sub-config]]
matches = "tests/**/*"
[tool.pyrefly.sub-config.errors]
missing-attribute = "ignore"

[[tool.ty.overrides]]
include = ["tests/**"]
rules = { unresolved-attribute = "ignore", invalid-assignment = "ignore" }

But scope the suppressed rule names narrowly (unresolved-attribute/missing-attribute/the specific invalid-assignment shape), not broad categories like invalid-argument-type/bad-argument-type - those catch genuinely wrong types passed into calls, which do happen in test code (typos, copy-paste of the wrong fixture) and are worth keeping visible. In the fastkml migration, roughly 90% of test-file errors were narrowing noise safely bulk-suppressed, and the remaining 10% surfaced one genuine test bug (a string literal assigned where an enum member was expected) plus a batch of deliberately-invalid-input tests that just needed their ignore-comment syntax migrated (see pitfall 1).

Phase 3 - Work file by file for what's left

After the systemic fix and the bulk test-suppression, what remains is usually a short, tractable list (tens, not hundreds, of diagnostics). For each:

1. Read the surrounding code before deciding how to fix it. The same unresolved-attribute shape can be a real narrowing gap (add assert x is not None, matching the style already used elsewhere in the file), a genuine latent bug (an off-by-one/empty-tuple case an unpacking *args call didn't guard against), or a structural dispatch limitation (pitfall 5).
2. Prefer a real fix over a cast or ignore wherever one exists cheaply: correcting a wrong return-type annotation (Optional[X] that never actually returns None), adding isinstance narrowing instead of a loose dict-dispatch, genericizing a find/find_all-style utility with @overload + a TypeVar instead of returning object.
3. Use cast() with a one-line comment explaining *why* when the limitation is structural (pitfalls 4-6), not because you're in a hurry.
4. Re-run the checker on just that file after each fix (ty check path/to/file.py) - faster feedback loop than re-running the whole tree, and it confirms the fix didn't introduce a new diagnostic in the same file.

Phase 4 - Tighten to "maximum quality"

Once both tools report zero errors on the fixed baseline, raise the bar deliberately rather than assuming the default preset is already strict:

[tool.pyrefly]
preset = "strict"   # not "legacy" / "default" - legacy exists specifically to ease mypy migrations, it's a floor, not a target

[tool.ty.rules]
# Promote rules ty ships at warn/ignore by default; discover the full list with `ty explain rule`.
possibly-missing-attribute = "error"
possibly-missing-import = "error"
possibly-unresolved-reference = "error"
missing-type-argument = "error"
unused-ignore-comment = "error"
unused-type-ignore-comment = "error"
redundant-cast = "error"

Discover what's available rather than guessing:

ty explain rule                 # every rule, default level, rationale, examples
pyrefly check --help            # --preset options, --error/--warn/--ignore, --replace-imports-with-any, etc.
pyrefly dump-config             # what's actually active for this project right now

Phase 5 - Verify

Don't call it done on "the type checker is quiet." Run the full loop:

ty check <src> <tests>
pyrefly check <src> <tests>
ruff check --no-fix <src> <tests>          # type-fix edits (casts, isinstance, overloads) can introduce lint issues
ruff format --check <src> <tests>
python -m pytest                            # with optional runtime deps installed
uv pip uninstall <optional-dep>             # e.g. lxml
python -m pytest -m "not <slow-marker>"     # confirm the fallback code path still works
uv pip install -e ".[typing,<optional-dep>]"
python3 -c "import tomllib; tomllib.load(open('pyproject.toml','rb'))"

The "uninstall the optional dependency and re-run tests" step matters specifically because Phase 2's fix changes how the optional backend is typed, not just how it's imported - if the runtime fallback logic was touched at all while chasing type errors, this is the step that catches a broken fallback path before it ships.

Case study numbers (fastkml)

For calibration on what "a lot of noise, mostly one root cause" looks like in practice:

• Baseline: ty reported 237 diagnostics across source + tests; pyrefly reported 40 errors (with an already-too-permissive config suppressing 38 more).
• One import-site fix (Phase 2, aliasing the duck-typed Element Protocol and etree module to lxml's real stubs under TYPE_CHECKING) collapsed ty's source-only count from 47 to 13 in a single step.
• Total genuine bugs found and fixed in library source: 5 (a missed .getroot() call, an unguarded empty-tuple unpack in a zip_longest loop, a too-loose type[object] dispatch signature, a Self-in-a-list invariance issue, and one example script passing bytes where str was expected).
• Total genuine bugs found and fixed in tests: 1 (a string literal compared against an enum field).
• Final state: zero errors on ty check and pyrefly check for the CI-covered scope, with all pre-existing tests still passing, both with and without the optional lxml backend installed.

If you made it this far, you might be interested in the cost:

• Total cost: $40.44
• Total duration (API): 58m 29s
• Total duration (wall): 1h 48m 15s
• Total code changes: 635 lines added, 224 lines removed

Usage by model:

• claude-haiku-4-5: 1.1k input, 39 output, 0 cache read, 0 cache write ($0.0013)
• claude-sonnet-5: 23.1k input, 227.0k output, 106.7m cache read, 993.5k cache write ($40.44)

05 Jul 2026 4:29pm GMT

I am pleased to announce the release of scikit-survival 0.28.0.

A highlight of this release is the support for Polars DataFrames alongside pandas DataFrames via the Narwhals dataframe abstraction layer. In addition, this release adds support for scikit-learn 1.9.

Support for Polars

Polars is a data frame library similar to pandas, but with its core written in Rust instead of Python, which often gives Polars an advantage in terms of performance.

All datasets shipped with scikit-survival can now be loaded as a Polars DataFrame by specifying the output_type argument.

from sksurv.datasets import load_gbsg2
# return X as a polars DataFrame
X, y = load_gbsg2(output_type="polars")
X.head()

shape: (5, 8)
┌──────┬────────┬───────┬──────────┬────────┬─────────┬────────┬───────┐
│ age ┆ estrec ┆ horTh ┆ menostat ┆ pnodes ┆ progrec ┆ tgrade ┆ tsize │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ enum ┆ enum ┆ f64 ┆ f64 ┆ enum ┆ f64 │
╞══════╪════════╪═══════╪══════════╪════════╪═════════╪════════╪═══════╡
│ 70.0 ┆ 66.0 ┆ no ┆ Post ┆ 3.0 ┆ 48.0 ┆ II ┆ 21.0 │
│ 56.0 ┆ 77.0 ┆ yes ┆ Post ┆ 7.0 ┆ 61.0 ┆ II ┆ 12.0 │
│ 58.0 ┆ 271.0 ┆ yes ┆ Post ┆ 9.0 ┆ 52.0 ┆ II ┆ 35.0 │
│ 59.0 ┆ 29.0 ┆ yes ┆ Post ┆ 4.0 ┆ 60.0 ┆ II ┆ 17.0 │
│ 73.0 ┆ 65.0 ┆ no ┆ Post ┆ 1.0 ┆ 26.0 ┆ II ┆ 35.0 │
└──────┴────────┴───────┴──────────┴────────┴─────────┴────────┴───────┘

Transforming DataFrames

scikit-learn enables transformers to return Polars data frames via the set_output API.

from sklearn import set_config
from sklearn.preprocessing import StandardScaler
set_config(transform_output="polars")
# standarize the columns of a polars DataFrame
X_standarized = StandardScaler().fit_transform(
X.select("age", "tsize")
)

scikit-surival has two transformers that now accept polars and pandas data frames as input: ClinicalKernelTransform, and OneHotEncoder.

ClinicalKernelTransform

ClinicalKernelTransform computes a kernel matrix, so the output will be a numpy array, as before. With scikit-survival 0.28.0, it is aware of the polars colum types String, Categorical, Enum and Object. However, polars does not have a concept similar to ordered categories in pandas: pd.Categorical([…], ordered=True).

When computing the clinical kernel for a polars data frame, you can specify the order of categories with the ordinal_categories argument, otherwise all non-numeric columns will be treated as nominal columns, where values have no specific order (e.g. the column horTh with values "yes" and "no" in the example below).

from sksurv.kernels import ClinicalKernelTransform
K = ClinicalKernelTransform(
ordinal_categories={"tgrade": ["I", "II", "III"]},
).fit_transform(
X.select("age", "horTh", "tgrade")
)

OneHotEncoder

OneHotEncoder encodes the string-type columns to numeric columns. It automatically returns a data frame of the same type as the input:

from sksurv.preprocessing import OneHotEncoder
X_onehot = OneHotEncoder().fit_transform(X.select("horTh", "tgrade"))
X_onehot.head()

shape: (5, 3)
┌───────────┬───────────┬────────────┐
│ horTh=yes ┆ tgrade=II ┆ tgrade=III │
│ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ f64 │
╞═══════════╪═══════════╪════════════╡
│ 0.0 ┆ 1.0 ┆ 0.0 │
│ 1.0 ┆ 1.0 ┆ 0.0 │
│ 1.0 ┆ 1.0 ┆ 0.0 │
│ 1.0 ┆ 1.0 ┆ 0.0 │
│ 0.0 ┆ 1.0 ┆ 0.0 │
└───────────┴───────────┴────────────┘

Fitting survival models

Finally, you can pass a polars data frame to the fit and predict functions of any estimator.

from sklearn.pipeline import make_pipeline
from sksurv.linear_model import CoxPHSurvivalAnalysis
pipe = make_pipeline(
OneHotEncoder(), StandardScaler(), CoxPHSurvivalAnalysis()
)
pipe.fit(X[:500], y[:500])
risk_scores = pipe.predict(X[500:])

Note that this only works for eager data frames, lazy data frames will give an error.

X_lazy = X.lazy()
pipe.fit(X_lazy[:500], y[:500])

Traceback (most recent call last):
File "example.py", line 21, in <module>
pipe.fit(X_lazy[:500], y[:500])
File "…/site-packages/sklearn/base.py", line 1403, in wrapper
return fit_method(estimator, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
…
File "…/site-packages/sksurv/_dataframe/_input.py", line 96, in ensure_eager_dataframe
_reject_polars_lazyframe(obj)
File "…/site-packages/sksurv/_dataframe/_input.py", line 84, in _reject_polars_lazyframe
raise TypeError(_LAZYFRAME_NOT_SUPPORTED_MSG)
TypeError: polars.LazyFrame is not supported; call .collect() before passing to scikit-survival.

New Contributors

A big shoutout to our two new contributors:

• cakedev0 for adding support for scikit-learn 1.9
• 55Kamiryo for adding support for Polars data frames.

Updated Dependencies

With this release, the minimum supported version are:

Install

scikit-survival is available for Linux, macOS, and Windows and can be installed either

via pip:

pip install scikit-survival

or via conda

 conda install -c conda-forge scikit-survival

05 Jul 2026 4:22pm GMT

A very strange Pi issue sent me down a rabbit hole over the last two days. The short version is that newer Claude models sometimes call Pi's edit tool with extra, invented fields in the nested edits[] array. And not Haiku or some small model: Opus 4.8. The edit itself is usually correct but the arguments do not match the schema as the model invents made-up keys and Pi thus rejects the tool call and asks to try again.

That alone is not too surprising as models emit malformed tool calls sometimes. Particularly small ones. What surprised me is that this is getting worse with newer Anthropic models as both Opus 4.8 and Sonnet 5 show it but none of the older models. In other words, the SOTA models of the family are worse at this specific tool schema than their older siblings.

In case you are curious about Fable: I intentionally did not test it because I was not sure if the classifiers they are running might downgrade me to Opus silently.

Tool Calls Are Text

If you have not spent too much time looking at LLM tool calling internals, the important thing to understand is that tool calls are not magic and use some rather crude in-band signalling. The model receives a transcript, a system prompt and a list of available tools. The server munches that into a large prompt with special marker tokens. Because the model was trained and reinforced on examples of that format, at some point during generation it emits something that the API or client interprets as "call this tool with these arguments".

For a file edit tool, the intended invocation payload might say something like this:

{
  "path": "some/file.py",
  "edits": [
    {
      "oldText": "text to replace",
      "newText": "replacement text"
    }
  ]
}

A harness then validates the arguments, performs the edit, and feeds the result back into the model. If validation fails, the model sees an error and usually tries again.

How exactly that formatting happens is not known for the Anthropic models, but some people have gotten out "ANTML" markers and they at times do leak also into public communications. To the best of my knowledge, the call above would come out serialized like this from the model:

<antml:function_calls>
  <antml:invoke name="edit">
    <antml:parameter name="path">some/file.py</antml:parameter>
    <antml:parameter name="edits">
[
  {
    "oldText": "text to replace",
    "newText": "replacement text"
  }
]
    </antml:parameter>
  </antml:invoke>
</antml:function_calls>

An important thing to note here is that this thing, while looking like XML, is not really XML. It's just a thing they found convenient to tokenize and train on. The other thing to note is that a basic top-level string parameter appears in-line whereas an array of objects is implemented via JSON serialization. While I'm not entirely sure that this is how it works, there are some indications that this is not too far off. This will become relevant later.

There are two very different ways to make the model produce a structure like this:

1. You can ask the model to produce valid JSON matching a schema and then validate it afterwards.
2. You can constrain the sampler so that invalid JSON, or even invalid schema shapes, cannot be sampled in the first place.

The second approach is what people usually refer to as grammar-aware or constrained decoding. The sampler masks out tokens that would violate the grammar. If the model is currently inside a JSON object and the schema says only oldText and newText are allowed, the sampler can prevent it from emitting "in_file" or "type". Grammar-aware decoding can be used both to constrain something to be syntactically valid JSON and also to enforce specific enum values or keys.

Without any form of constraints the model is merely following a learned convention.

The Failure

Pi's edit tool supports multiple exact string replacements in one call. That is why the arguments contain an edits array. In the failing cases the model produces entries like this:

{
  "oldText": "...",
  "newText": "...",
  "requireUnique": true
}

or this:

{
  "oldText": "...",
  "newText": "...",
  "oldText2": "",
  "newText2": ""
}

Across repeated trials I saw a whole zoo of invented trailing keys: type, id, kind, unique, requireUnique, matchCase, in_file, forceMatchCount, children, notes, cost, oldText2, newText2, oldText_2, newText_2, and even an event.0.additionalProperties key inside the edit object itself.

The most annoying part is that the actual oldText and newText payloads were byte-correct in the invalid calls I inspected. The model had in fact produced the right invocation but then added nonsense at the end of the object.

The failure is also heavily context-dependent. A fresh single-turn prompt like "edit this file" did not reproduce it at all for me. An agentic history where the model had read files, diagnosed a problem and then composed a multi-line edit could reproduce it. And more annoyingly, not all transcripts will show that behavior. In fact, I needed Petr Baudis's transcripts to reproduce this for me at all! In that user's session continuing the session caused Opus 4.8 to fail around 20% of the time. Stripping thinking blocks from history reduced the failure rate by half. Turning on strict tool invocation eliminated it in my runs.

Why It's Getting Worse

My strongest hypothesis is that this is not random deterioration but a training artifact.

When older Anthropic models were trained, they were trained on some tools (some of which were documented). But that training did not yet have a user-shipped harness like Claude Code as the obvious target. Modern Anthropic models are most likely different because their post-training includes Claude Code or a harness that looks very similar. The model learns what a successful tool call looks like in that environment. It also learns what mistakes are tolerated by that environment.

Claude Code's own tools are comparatively flat. The ordinary edit tool is not Pi's nested edits[] shape; it is closer to file_path, old_string, new_string, and an optional flag (replace_all). Looking at Claude Code's client is very instructive: it contains retry paths for malformed tool use, parameter aliases, type coercions, Unicode repairs and filtering of unknown keys. In other words, Anthropic's own client appears to expect and accept a fair amount of slop and repairs it, mostly silently.

If reinforcement learning happens in a harness like that, or a simulation of one, then slightly malformed tool calls can still complete the task and receive reward. The harness fully absorbs the error and there is little gradient against inventing an alias, adding a stray field or using a nearby parameter name.

Worse, the model may become very strongly adapted to the canonical Claude Code edit tool shape. A different harness can present a tool with the same semantic intent but a different schema. Such a tool can increasingly be off-distribution. The better-trained model might actually fight you harder because its prior is stronger.

This is not too surprising, but it is a change from how this was a few months ago. When Opus 4.5 launched, it adapted to other edit tools exceptionally well. In fact, I was pretty convinced that we're on a good path where the models are more likely to adapt to any sort of tool shape that comes around for as long as the instructions are good.

Now I'm somewhat worried about the track we're on here. Alternative tool schemas might not just be unfamiliar. They might be implicitly punished by post-training that optimizes for one particular, forgiving tool ecology. And that ecology is not documented. While there is a text editor tool that is documented, you will see that this format is in fact not followed by Claude Code. What Claude Code does internally (which is a closed-source harness) is hidden from you.

The Slop Harness

Claude Code is obviously closed-source but we can look at the minified code and get some idea of what it does. And honestly, it's very forgiving of incoming data.

For a start, Claude Code checks the model's visible text for leaked <invoke markup. It also emits some telemetry when that happens and then it has its own state machine to retry such bad calls by pushing back to the model.

It has explicit Unicode escape repair which fixes broken \uXXXX sequences and lone surrogates in string values. It also has per-tool aliases for parameters. For instance, Edit accepts old_str (presumably from the times when the models were trained on the officially documented text editor tool), the newer old_string from the schema, new_str/new_string, path as an alias for file_path, and some more.

It also silently filters out unexpected keys and it does not use strict mode either. The issue with strict mode is that Anthropic applies complexity limits to the tool definitions that cause API requests to fail, so presumably that's why Claude Code does not attempt to use it.

Strictness

Will this problem be with us in other harnesses too? One huge issue with Anthropic is that the models are completely closed, and so is the harness. Codex models are also closed, but at least the harness is not. We also have gpt-oss which is at least a bit interesting. The models are explicitly trained to use OpenAI's harmony response format and there is a lot of documentation that at least tells us how OpenAI people think about this.

Harmony makes channels and tool-call content types part of the prompt format. A function call can look like this:

<|start|>assistant<|channel|>commentary to=functions.get_weather
<|constrain|>json<|message|>{"location":"San Francisco"}<|call|>

The important bit is <|constrain|>json. The model can express in-band that this message body is JSON, and an inference stack can use that boundary to switch into JSON-constrained sampling for the body of the tool call. Presumably a bit of this also happens in Anthropic's models, at least in strict mode I would imagine.

The marker in harmony helps the sampler to detect when it needs to sample with a specific grammar, and because it is part of the transcript, it makes that rather easy to do. For hosted GPT models, there is also an option to provide a LARK grammar for custom tools that need to adhere to something like this.

Anthropic appears different from that, though maybe not entirely. If an array of objects is represented as JSON, as it appears to be, then the model has to write JSON inside the tool parameter. There is probably basic grammar-constrained sampling going on, and that may partly explain the extra keys. For a nested array parameter, that JSON includes escaped multi-line file content inside string literals, inside one tag. The unexpected, made-up keys appear exactly at the highest-entropy point of that task: after closing a several-hundred-token escaped newText string, where the model must decide } vs , "...".

Opus 4.8 and Sonnet 5 seem to have much stronger priors about what an edit tool call should look like and that prior appears to be Claude Code's edit schema: a flat old/new string pair, plus the optional replace_all flag. My guess is that Opus has learned that an edit operation may have one extra optional field, but under Pi's nested oldText/newText shape it has no trained name for that field. So it samples a plausible name fresh each time, which is why the failures produce dozens of random keys rather than one stable alias.

As strict mode in Anthropic appears to fix this, I presume that on the server side they are refusing to sample a key that is not permitted by the JSON schema structure. That would also explain why they have limits to the complexity of the tool definitions when strict mode is enabled.

So far, the Codex models I tested did not show this type of regression. I tested all available ones except 5.6, which I do not have access to yet.

What This Means For Harnesses

The uncomfortable lesson is that tool schemas are not neutral, at least not on Anthropic models. We like to pretend that a schema is an abstract contract and the model is a general reasoner that will follow it, but that might no longer be the case for some of the tools.

Tool schemas are somewhere in the distribution and some shapes are close to what the model saw during post-training and some are far away. Some are easy for the provider's hidden encoding (e.g. top-level attributes in ANTML), whereas some require the model to write large escaped JSON objects inside nested arrays after long multiline strings. The model may be smart enough to understand the schema and still be bad at sampling the exact shape under pressure.

If this type of model behavior continues, I wonder what the implications for harnesses are. Obviously one could turn on strict sampling in Anthropic and the problem should go away. On the other hand, that the model has this behavior shows the impact that reinforcement learning has on them. Fighting that prior is probably futile if you want to get the best model performance.

Right now the reality is that Claude Code is not open source and we cannot really know what they are doing in their RL environments either. We cannot assume Claude-Code-trained behavior will transfer cleanly to your tools unless they are a close match. The more post-training happens inside one dominant harness, the more every other harness will have to inherit its quirks.

I used to be more skeptical of strict grammar-constrained tool invocation because constrained decoding can have quality tradeoffs. I still think that can be true in general, but this bug moved my priors significantly. If the newest models get better at solving the task while getting worse at faithfully emitting an alternative tool schema, then the harness needs stronger guarantees somewhere.

If you want to find out more, or you want to discuss this, consider reading the issue on the Pi tracker.

04 Jul 2026 12:00am GMT

03 Jul 2026 3:00pm GMT

import typer

app = typer.Typer()


@app.command()
def hello(name: str):
    print(f"Hello {name}")


if __name__ == "__main__":
    app()

02 Jul 2026 4:00am GMT

Weeknotes (2026 week 27)

The last entry in this series was published 10 weeks ago so it really is time for another review of the releases I did during this time.

Releases

feincms3-forms

The feincms3-forms forms builder has gained a documentation page on the wonderful Read the Docs service. The 0.6.1 release doesn't contain any code changes, just pyproject.toml updates and the mentioned documentation rework.

django-imagefield

django-imagefield 0.23 is still in alpha. The handling of image fields when using libvips is optimized to use less memory hopefully. We'll see. I also added some tests to verify that .mpo files are handled properly.

feincms3

The Vimeo embed now always sets the dnt=1 parameter on the <iframe>, which asks Vimeo to not track the user.

django-mptt

I wrote about the somewhat annoying maintenance again. The library is still officially unmaintained, but I did a lot of work either just closing issues or also fixing them. The docs also contain many clarifications. I only released 0.19rc1 for now.

feincms3-sites and feincms3-language-sites

Last time I mentioned that default HTTP/S ports are now stripped so that the host matching can determine the correct site. Now a new case appeared where trailing dots weren't stripped. The normalization of hosts has been extended. I'm sure we're still missing some exotic cases where we should do more normalization, but we'll cross that bridge when we get there.

django-prose-editor and django-js-asset

Various upgrades to the editor and especially the importmaps rework in both packages - the importmap infrastructure should now be CSP-compatible! I wrote more about that in the last post The 2026 way of using importmaps in Django.

django-content-editor

Minor bugfixes and a major version bump because of the rework of the JavaScript code into multiple ES modules. The content editor now uses importmaps as well.

django-fhadmin

Small bugfix so that links aren't underlined in the app groups list when they shouldn't be, matching how the Django admin itself behaves.

django-cabinet

The cabinet / prose editor integration for the file (or image) picker is final and released as a stable version.

django-json-schema-editor

This small release only contains more correct German translations of strings.

Honorable mention: django-debug-toolbar

I didn't actually create this release, but I contributed various changes to it. The changelog for 7.0 is here.

01 Jul 2026 5:00pm GMT

As I have discussed in previous posts, "AIs" can make mistakes. In fact, they do make mistakes, and their mistake-making patterns are such that where and how they will make mistakes is both uncertain and constantly changing.

Thus, in any scenario where you want to attempt to make "productive" use of "AI", you must have a system in place for checking every result. Not checking some results; checking every result. If each result might have a consequence for you (and if it didn't have a consequence, why bother automating it?) and you cannot predict in advance which kinds of results will need verification, then verification is always required.

The verification often ends up being just as expensive as doing the work in the first place, which means that if you want your usage of "AI" to be personally profitable, you have to find someone else to externalize the cost of verification onto. This person becomes your adversary, and, if you are successful, your "AI's" victim.

The Ladder-Climber And Their Reverse-Centaur Rungs

One way that this constellation of facts can straightforwardly assemble themselves into a dystopian nightmare is the phenomenon, described by Cory Doctorow, of the reverse centaur. This is when your employer non-consensually turns you into the verification system. The "AI" does the fun part of initially performing the work, and then you do the boring part where you check if the robot is right and clean up its messes, even if everyone already knows that it would, in aggregate, be cheaper for you to do the work in the first place.

Reverse centaurs can be made from any automation, not only "AI" automation. I think that there is a reason that this term happens to have emerged in the "age of AI", though, and not with earlier automation technologies (even those which were considerably more viscerally horrific). That reason is: the wrongness of "AI" output is not merely a technical feature that must be compensated for, it is a generalized externality.

As I mentioned above, if you are responsible for the entirety of the work, both extruding the "AI" output and checking it, it's usually cheaper to have humans do the entirety of the work to begin with. When humans do the writing directly, we can check as we go, and thus verification doesn't need to be as comprehensive.

When "AI" coding advocates say "code review is the bottleneck", what they are observing is that the LLM is still rolling the dice for each PR, and a human is still necessary to verify that each of those rolls is a winner. But calling this process "code review" is a bit of a misnomer; it's not really "code review" in the traditional sense, it's human understanding.

Before the advent of "AI", the human understanding was implicit in the process of writing the code in the first place¹, and the code review was a way of diffusing and extending that understanding. Now that the code can be authored with no initial understanding taking place, that cost has not gone away, it has moved.

Human understanding was always the bottleneck.

However, this is taking a collaborative view of a software project, where satisfying the needs and solving the problems of your customers are the goals. We can see that "AI" is a bad tool to satisfy those goals, because all it's doing is converting the first half of the work, that of understanding the code as you write it, to understanding the agent's output as you read it.

What if, instead, we were to take the view that every software company is a Hobbesian nightmare, red in tooth and claw? In this view, the only goal of a software project is for the individual developers to make their promo cycles and get their bonuses. Given that there is only a certain amount of money to go around, this is a zero-sum game where each programmer wants to look more productive than their colleagues.

Pretty much every organization finds it easy to reward "productivity" as expressed by lines of code emitted, but the benefits of doing thorough and thoughtful design, analysis, and code review very difficult to reward. In this world, an LLM is an invaluable tool for the sociopathic ladder-climber, particularly if your legacy organization is still structuring their workflows as if the person prompting the bot is "writing" the code, and then they get to foist off the act of "reviewing" the code onto someone else.

Here, the prompter effectively externalizes the cost of the LLM's failures but internalizes any benefits. The prompter will vibe-code a big feature, so large that the assigned reviewer can't possibly comprehend it all effectively. When this happens, the reviewer will, eventually, be pressured to approve it, even if they can try to spot a few problems along the way. The reviewer has their own work to get back to, after all, the obligation to review the prompter's (read: the bot's) code is a drain on their time that they are not going to get rewarded for.

If this feature is a big success, the prompter gets a promotion. If it causes a big issue, well, the reviewer must not have been careful enough.

This is why LLMs are "good for coding", and also why their biggest promoters keep having outages.

The Generative Gish Galloper

Coding is the biggest "success story" of this type of adversarial communication, but it is by far not the only instance of such a thing. LLMs create a new form of leverage that can turn Brandolini's law from a linear advantage into an exponential one. If you are engaged in a political debate where you want to overwhelm the other side in nonsense, an LLM can generate bullshit faster than it is physically possible for a human being to type, let alone respond thoughtfully. There is an asymmetry to the utility of this weapon as well: only one side of the political spectrum wants to flood the zone and destroy trust in institutions and the concept of truth. There's a good reason that the fascists love it.

Straightforward Spam and Fraud

This is kind of obvious, but LLMs can generate lightly-customized, plausible-looking text much more quickly than any human being. This facilitates their use in fraud, spam, and scams. In a spamming or fraudulent interaction, once again, the costs are externalized onto the victim: the recipient of a spam message has to do all the work of "checking" the LLM's output. Spammers already expect very low hit rates from boilerplate, and if the LLM can increase those percentages from 1% to 5% the technology will pay for itself; they don't need anything like reliable accuracy.

Customer "Support"

If you have any kind of commercial relationship with a company, I probably don't even need to mention this: customer "support" bots are a misery. Everybody knows it at this point. But customer support is usually conceptualized by businesses as an adversarial interaction, because it is a cost center. They maintain internal metrics on time-to-resolution and try to optimize them. Implicitly, this creates a dynamic where the goal of the customer service agent's job is not to solve your problem, but to emit noise that will cause you to think your problem is resolved, or to give up, as fast as possible. Unsurprisingly, LLMs can emit this noise faster than humans can, getting those customers off the phone. But those customers will remember those interactions, and the story outside the TTR metrics is horrible.

Similarly to the situation in software development, LLMs can look very good on paper for customer support, but mostly what they are doing is illuminating the problems with the industry's existing metrics, by turning "winning the metrics battle against the customer" into a more obvious and immediate defeat for the company's long term reputation.

"Education"

In 2026 it is sadly a fact of life that students cheat all the time using "AI", and that this cheating is very successful, in that the teachers find it very hard to detect.

LLMs are great for cheating on schoolwork because the student is externalizing the work of the checking onto the teachers, who are often starting at a disadvantage to begin with, at least in the US.

My view is that this is happening because of a divergence in the way that students vs. teachers (or, more accurately, "the broader educational system") view grading.

When a student is asked to write an essay, the teachers see the effort as both intrinsically worthwhile for the student, as well as useful as a pedagogical tool to evaluate and react to the student's progress. The student, by contrast, sees a stumbling block designed to knock them off the path to success and into a permanent underclass. It is no wonder that the student sees "AI" as useful to their own goals and has no compunction about deploying it.

There is a bitter irony that the ability to understand the inherent value of actually writing the essay on their own is the sort of thing that students can really only learn by writing a bunch of essays. There's no way that I can think of which makes the benefit legible as long as a shortcut is available.

The net effect here is a downward spiral, where the already-wobbling educational system is sustaining an attack that it doesn't have the resources to recover from. The individual students' attacks against their teachers and their schools' grading systems might appear to momentarily succeed, but they will win the battle and lose the war.

Spamming "For Good"?

Usually when we talk about someone unilaterally choosing to enter into an adversarial relationship, that's an "attack" and for good reasons we have a negative impression of the attacker. However, I would be remiss if I did not point out that there are some cases where the relationship was already adversarial; just because you're the attacker doesn't mean that you are evil.

For example we might imagine use-cases like automatically filing appeals for prior authorizations against health insurance. It's relatively well-known at this point that the main way for-profit insurers maintain their margins is by denying claims right up to the line of the policies themselves being fraud, so using a spamming tool to fight them might be entirely justifiable² in that case.

Similarly, using an LLM could be justified in a fight against a company refusing to honor a warranty. One could imagine using an LLM to immediately generate replies and escalations.

However, even in imagined cases like these, the underlying problem is that the insurers and the vendors already have a tremendous amount of structural power, so it is more likely that they will have the advantage in deploying a communications weapon like an LLM, as well as enacting policies to simply ignore any LLM-based communication that you might submit. Worse, if these strategies were to become widespread, they might provide an excuse to reject any communications by feeding them into an unreliable "LLM detector" and issuing an automated "computer says no" even to hand-written correspondence.

It is also worth stressing that these cases are imagined, as compared to the very real coworker-abuse, spam, scam, fraud, and disinformation campaigns being waged in real life today.

Therefore, while legitimate uses might exist, it's hard to imagine that there's anywhere they would be genuinely valuable and sustainable. In the best case "AI" will provide a temporary advantage for underdogs that will provoke an arms race which the resource-advantaged adversaries will win in the long run, in the worst case the arms race itself will cement permanent structural change that will make things worse.

"Search" By Stealing

Most of the adversarial utility of "AI" is on the "write" side, since write-amplification is more obviously aggressive than reading. But the "read" side of LLMs - summarization and question-answering - can be a form of attack as well.

To begin with, the act of reading itself is currently enormously destructive, but that's arguably not a fundamental aspect of this technology. They could set reasonable rate-limits and respect things like robots.txt, as search engines have for decades now. They could also refrain from committing criminal levels of copyright infringement. But, today, using "AI" tools does suborn this sort of out-of-control crawling.

More insidiously, consider the scenario described in this YouTube video. The LTT Bros decided to try Linux again, and in the course of so doing, they had problems. When trying to solve these problems, they were faced with a choice: they could consult Reddit, or they could ask an LLM. Asking an LLM would "gaslight the heck out of" them, but they still found it preferable, because they would at least get an answer without getting yelled at.

Initially this sounds great. But it also means that you want to extract knowledge from a community, while mechanically eliding any values or norms that the community may want to impart as part of offering that knowledge. As someone who spent many years in a community tech support role, this is worrying. Many requests for support are people asking how to do things that will momentarily solve a superficial problem but create a long-term reliability problem or even an immediate security risk, that the question-asker doesn't want to hear about. Consider the question "I'm tired of entering my password so much, how do I make it so my laptop unlocks automatically". An obsequious chatbot will helpfully tell you how to do this without pushback.

But, this is also a sort of ethically murky area. The Linux community is somewhat famously, for many years now, a toxic cesspool of general hostility, misogyny, etc. It is certainly a good thing that people can get access to this knowledge without subjecting themselves to abuse. But it also means that the people with the power and the privilege to change the community for the better can just quietly withdraw, rather than fixing the problems. It also means that the positive elements of culture cannot be transmitted, and people will have no opportunity to learn about unknown unknowns.

In this case, the "adversarial" communication is with society. The thing that using an LLM for search lets you do is withdraw from society and avoid forming any personal connections. There are some personal connections which are painful and annoying, and so that can feel like a momentary balm. But the need to make connections in general is, like, the concept of society itself.

Who Am I Hurting?

LLMs are good at adversarial communication. They are so good at it, relative to their other benefits, that they will tend to make communications adversarial if you are not remaining vigilant about the possibility that it might do so. My request to you, dear reader, if you are going to use such tools, is to always ask yourself, "who might I be hurting, if I use an LLM for this?"

If you're using an "AI", who is its adversary? If you haven't given it one yet, who might the "AI" turn into an adversary? Who might you overwhelm with an asymmetric amount of output, or, if you're receiving information and not sending it, who are you taking that information from without consulting?

Figure out the answers to these questions and conduct yourself accordingly; the answer might be "yourself".

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!

23 Jun 2026 8:06pm GMT

Codecov's unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn't rely on third-party services.

09 Jun 2026 12:00am GMT

Let's say you're writing a Python library.

In this library, you have some collection of state that represents "options" or "configuration" for a bunch of operations. Such a set of options is a bundle of potentially ever-increasing complexity. Thus, you will want it to have an extremely minimal compatibility surface, with a very carefully chosen public interface, that is either small, or perhaps nothing at all. Such an object conveys state and might have some private behavior, but all you want consumers to be able to do is build it in very constrained, specific ways, and then pass it along as a parameter to your own APIs.

By way of example, imagine that you're wrapping a library that handles shipping physical packages.

There are a zillion ways to do it ship a package. There are different carriers who can ship it for you. There's air freight, and ground freight, and sea freight. There's overnight shipping. There's the option to require a signature. There's package tracking and certified mail. Suffice it to say, lots of stuff.

If you are starting out to implement such a library, you might need an object called something like ShippingOptions that encapsulates some of this. At the core of your library you might have a function like this:

1
2
3
4
5

async def shipPackage(
        how: ShippingOptions,
        where: Address,
    ) -> ShippingStatus:
    ...

If you are starting out implementing such a library, you know that you're going to get the initial implementation of ShippingOptions wrong; or, at the very least, if not "wrong", then "incomplete". You should not want to commit to an expansive public API with a ton of different attributes until you really understand the problem domain pretty well.

Yet, ShippingOptions is absolutely vital to the rest of your library. You'll need to construct it and pass it to various methods like estimateShippingCost and shipPackage. So you're not going to want a ton of complexity and churn as you evolve it to be more complex.

Worse yet, this object has to hold a ton of state. It's got attributes, maybe even quite complex internal attributes that relate to different shipping services.

Right now, today, you need to add something so you can have "no rush", "standard" and "expedited" options. You can't just put off implementing that indefinitely until you can come up with the perfect shape. What to do?

The tool you want here is the opaque data type design pattern. C is lousy with such things (FILE, pthread_*_t, fd_set, etc). A typedef in a header file can easily achieve this.

But in Python, if you expose a dataclass - or any class, really - even if you keep all your fields private, the constructor is still, inherently, public. You can make it raise an exception or something, but your type checker still won't help your users; it'll still look like it's a normal class.

Luckily, Python typing provides a tool for this: typing.NewType.

Let's review our requirements:

1. We need a type that our client code can use in its type annotations; it needs to be public.
2. They need to be able to consruct it somehow, even if they shouldn't be able to see its attributes or its internal constructor arguments.
3. To express high-level things (like "ship fast") that should stay supported as we add more nuanced and complex configurations in the future (like "ship with the fastest possible option provided by the lowest-cost carrier that supports signature verification").

In order to solve these problems respectively, we will use:

1. a public NewType, which gives us our public name...
2. which wraps a private class with entirely private attributes, to give us an actual data structure, while not exposing the constructor,
3. a set of public constructor functions, which returns our NewType.

When we put that all together, it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

from dataclasses import dataclass
from typing import Literal, NewType

@dataclass
class _RealShipOpts:
    _speed: Literal["fast", "normal", "slow"]

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("fast"))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("normal"))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("slow"))

As a snapshot in time, this is not all that interesting; we could have just exposed _RealShipOpts as a public class and saved ourselves some time. The fact that this exposes a constructor that takes a string is not a big deal for the present moment. For an initial quick and dirty implementation, we can just do checks like if options._speed == "fast" in our shipping and estimation code.

However, the main thing we are doing here is preserving our flexibility to evolve the related APIs into the future, so let's see how we might do that. For example, let's allow the shipping options to contain a concrete and specific carrier and freight method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

from dataclasses import dataclass
from enum import Enum, auto
from typing import NewType

class Carrier(Enum):
    FedEx = auto()
    USPS = auto()
    DHL = auto()
    UPS = auto()

class Conveyance(Enum):
    air = auto()
    truck = auto()
    train = auto()

@dataclass
class _RealShipOpts:
    _carrier: Carrier
    _freight: Conveyance

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.FedEx, Conveyance.air))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.UPS, Conveyance.truck))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.USPS, Conveyance.train))

def shippingDetailed(
    carrier: Carrier, conveyance: Conveyance
) -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(carrier, conveyance))

As a NewType, our public ShippingOptions type doesn't have a constructor. Since _RealShipOpts is private, and all its attributes are private, we can completely remove the old versions.

Anything within our shipping library can still access the private variables on ShippingOptions; as a NewType, it's the same type as its base at runtime, so it presents minimal¹ overhead.

Clients outside our shipping library can still call all of our public constructors: shipFast, shipNormal, and shipSlow all still work with the same (as far as calling code knows) signature and behavior.

If you need to build and convey some state within your public API, while avoiding breakages associated with compatibility churn, hopefully this technique can help you do that!

Acknowledgments

Thanks for reading, and thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor.

22 May 2026 12:33am GMT