26 Jun 2026

feedDjango community aggregator: Community blog posts

Open Source Comes From People

I recently attended my first PG Data 2026 conference where keynote speaker Robert Haas delivered a talk that has stayed with me. His keynote focused on the people behind PostgreSQL, the growing challenges of sustaining open-source communities, and the urgent need to cultivate new contributors through mentorship and community engagement. While his remarks centered on PostgreSQL, they sparked broader reflections for me about the future of open source and communities like Django.

26 Jun 2026 7:00pm GMT

Issue 343: Django 6.1 beta 1 released

News

Django 6.1 beta 1 released

Django 6.1 beta 1 is now available, giving the community a chance to test upcoming features and improvements before the final release on August 5.

Djangonaut Space: Launching Contributors

Djangonaut Space shares the results from its first six mentorship sessions, showing how an 8-week cohort program helped launch 104 contributors from 40+ countries into long-term open source participation and leadership.


Django Software Foundation

How the Django Software Foundation Became a CNA

Learn how the Django Software Foundation became a CVE Numbering Authority, giving it the ability to assign CVE IDs directly and streamline Django's security advisory process.


Wagtail CMS News

Wagtail as Django admin on steroids

Think Wagtail is just a CMS? See why it can serve as a polished, modern replacement for Django's admin with a familiar API and powerful features that make client-facing backends shine.

Comparing open weight AI models and providers

Open weight AI models are closing the gap with proprietary LLMs, and this guide explains how to compare models and providers on performance, cost, energy use, and transparency.


Releases

Python 3.15.0 beta 3 is here!

Python 3.15 beta 3 is out with nearly 200 bug fixes plus major additions like lazy imports, frozendict, sentinel objects, a faster JIT, and UTF-8 as the default encoding.


Updates to Django

Today, "Updates to Django" is presented by Raffaella from Djangonaut Space! πŸš€

Last week we had 24 pull requests merged into Django by 16 different contributors - including 2 first-time contributors! Congratulations to Margaret Fero and diaxoaine for having their first commits merged into Django - welcome on board!


Articles

Teach your linter your own rules

boa-restrictor is a Python/Django linter that now lets you register your own AST-based rule classes via pyproject.toml to enforce project-specific conventions. This is especially useful as a deterministic guardrail for keeping AI coding agents from repeating unwanted patterns.

Why I wrote PEP 832 -- virtual environment discovery

PEP 832 proposes a standard way for editors and AI tools to discover Python virtual environments, aiming to make project setup smoother regardless of your workflow tool.

Supporting Django's Next Chapter

Caktus Group has become a founding sponsor of the Django Software Foundation's new Executive Director position, investing in Django's long term sustainability and encouraging other companies to do the same.

Mitigated API authentication bypass for python.org download metadata

Python.org has disclosed and mitigated an authentication bypass that could have altered download metadata, with no evidence of exploitation after extensive audits and additional security hardening.

How I Architected Automatic Parking Detection in Django - Bluetooth Disconnects, Geofence Events, and a Strict State Machine

A deep dive into building a reliable Django parking detection system using Bluetooth events, geofencing, state machines, and optimistic locking to safely handle concurrency.

What I learned from two days of hanging out with AI experts

Five practical takeaways from an AI conference suggest the future belongs to model agnosticism, measurable ROI, and smaller open models instead of hype.


Videos

Learning Python in the Age of AI

In this short interview from PyCon US, Sheena O'Connell discusses one of the biggest questions facing developers today: how should people learn Python in the age of AI?

Paolo Melchiorre on AI-Assisted Development

Another PyCon US 2026 chat, this time with Paolo Melchiorre talking about Django, AI-assisted development, open-source maintainership, and how the Python community is adapting to AI.


Django Forum

Django 6.1 release - timeline and next steps

Notes and updates from Fellow Jacob Walls on the 6.1 release process.

Adding database backend methods to get hardcoded or nonexistent primary key values for tests

From Tim Graham, surfacing ticket #37175 "to see what our creative community can suggest."


Django Fellow Reports

Jacob Walls

Tended to a flurry of fixes before the non-release-blocker bugfix freeze for Django 6.1 in a few days. Also chipped away at some performance improvements for ASGI projects using sync middleware.

Natalia Bidart

Lots of preparation for the upcoming 6.1 Ξ²eta, with the goal of stabilizing recent changes and ensuring overall readiness πŸš€. I also spent time digging into Django's async behavior, reviewing recent changes and following through on related optimizations and documentation updates πŸ“’. I also looked more closely at packaging and reproducibility, especially around artifact builds, to improve our consistency in the release process πŸ“¦.


Django Job Board

Senior Python/Django Developer at Gryps

Founding ML/Data Scientist (Remote, UK) at MyDataValue


Projects

vintasoftware/django-ai-boost

A MCP server for Django applications, inspired by Laravel Boost.

Archmonger/ServeStatic

Production-grade Python static file server. Run as middleware or standalone.

26 Jun 2026 3:00pm GMT

feedPlanet Python

Bob Belderbos: There Is No Magic: An AI Agent in 60 Lines of Python

Everybody talks about agents, and a lot of people assume they're some new kind of model. They aren't. An agent is a small amount of plumbing around an LLM you already understand. Let's build one from scratch in Python and see exactly what that plumbing is.

The formula

An agent is: Model + Instructions + Memory + Tools + Execution Loop.

Five parts. None of them is magic. The model is a brain in a jar: useful, fast, but stateless. It generates text; the code around it decides what to do with that text. That second half is the entire job and it's code we can reason about.

I made the same argument about the control layer being the real product. Here it is as a program.

Start with the model. A real one calls an LLM API; we use a fake one that satisfies the same interface:

from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class Say:
    text: str

@dataclass(frozen=True)
class Call:
    tool: str
    arg: str

Reply = Say | Call

class Model(Protocol):
    def respond(self, system: str, history: list[str]) -> Reply: ...

The Model protocol has a single method, respond, which takes the system prompt and the conversation history and returns a Reply. It's a Protocol, so any object with a matching respond method counts as a Model, no inheritance required.

For this minimal agent, the Reply type captures the two actions we support: say something to the user, or call a tool with an argument. The model is free to return either one, and the agent will execute it. (Real models can also emit plans, ask clarifying questions, or request several tool calls at once; we keep it to two to stay legible.)

The agent's entire decision space is those two variants. The match in the loop below reads as a clean two-way branch, one case per reply, instead of a tangle of flags.

from dataclasses import dataclass, field
from typing import Callable

Tool = Callable[[str], str]

@dataclass
class Agent:
    model: Model                                          # 1. Model
    system: str                                           # 2. Instructions
    history: list[str] = field(default_factory=list)      # 3. Memory
    tools: dict[str, Tool] = field(default_factory=dict)  # 4. Tools

In this example, a tool is a function taking a string and returning a string. The agent holds the other four parts as plain fields:

The loop is the agent

The part that turns a well-instructed chatbot into something agent-like is the fifth piece: an execution loop that lets the model observe outcomes and decide what to do next. Observe, think, act, check, repeat. Greatly simplified, of course, but this is the piece that does the work.

Because the model is stateless, the agent must keep track of what happened and feed the history back into the model until the model decides the job is done.

    def run(self, user_input: str) -> str:
        self.history.append(f"user: {user_input}")
        while True:  # real agents cap the iterations; see termination guards below
            match self.model.respond(self.system, self.history):
                case Say(text):
                    self.history.append(f"agent: {text}")
                    return text
                case Call(tool, arg):
                    fn = self.tools.get(tool)
                    result = fn(arg) if fn else f"no such tool: {tool}"
                    self.history.append(f"tool[{tool}]: {result}")
                    # loop again: the model sees the result and decides what's next

Read it as the cycle:

There is no separate "check" block in the code. The check happens implicitly when the loop restarts and calls respond again with the new history. That step is the one that matters, because a model has no native sense of when a job is finished, and nothing stops it from asking for one more tool forever. The loop keeps going until the model returns Say instead of Call.

To run the whole thing without an API key, swap in a fake model and a real tool:

from pathlib import Path

def read_file(path: str) -> str:
    try:
        return f"{len(Path(path).read_text())} bytes"
    except OSError as e:
        return f"error: {e}"

class FakeModel:
    def respond(self, system: str, history: list[str]) -> Reply:
        last = history[-1] if history else ""
        if last.startswith("tool["):
            return Say(f"Done: {last}")
        if last.startswith("user: read "):
            return Call("read_file", last.removeprefix("user: read ").strip())
        return Say("I can read files. Try: read <path>")

Wire it into a small main that builds the agent, reads a line, calls agent.run, and prints the reply:

def main() -> None:
    agent = Agent(
        model=FakeModel(),
        system="You can read files.",
        tools={"read_file": read_file},
    )
    while True:
        try:
            line = input("> ")
        except EOFError:
            break
        print(agent.run(line.strip()))

if __name__ == "__main__":
    main()

Now you can talk to it with no API key. Run it with python agent.py and type at the prompt:

> read pyproject.toml
Done: tool[read_file]: 76 bytes

That one exchange is a complete agent loop: the model asked for a tool, the loop ran it, fed the byte count back, and the model wrapped up on the second pass. The main thing standing between it and a real one is replacing FakeModel.respond with an HTTP call that returns the same Reply.

The whole thing as one runnable file is here as a GitHub gist. Save it, run python agent.py, and type at the prompt.

What this earns you

Sure, this is a simplified example, and the hard parts are exactly what FakeModel stubs out: prompt design, retries, tool schemas, context compaction, error recovery, and termination guards that stop the loop when a model keeps hallucinating tools. But the core of an agent is 60 lines and easy to reason about. The engineering lives in the control layer around the model.

Build the loop by hand once and frameworks stop feeling magical. LangChain's agent executor, AutoGen's shared memory, a coding agent's plan mode are all variations on these same five parts: engineering tradeoffs, not magic.

Keep reading

26 Jun 2026 12:00am GMT

25 Jun 2026

feedPlanet Python

Artem Golubin: Hexora v0.3: New features and improvements

Recently, I've improved my Python library, hexora. I wrote it to detect malicious Python code using static analysis.

In the new v.0.3.0 release, I've added new detections, and we now also use a simple machine learning model to analyze the whole file. The machine learning model uses code structure features, semantic features, and static code analysis to assess the entire Python file.

Although the model can detect malicious code without any detections coming from static analysis, its main use case is to filter false positives.

I've been testing it against newly published PyPI packages and it detects 2-10 new malicious packages each day.

Due to the number of published packages, before the machine learning model, I was getting around 5-10 false positives for 1[......]

25 Jun 2026 1:37pm GMT

Django Weblog: How the Django Software Foundation Became a CNA

Why the DSF pursued CNA status

Django has a long history of responsible security practices: a dedicated, private security mailing list, clear advisory policies, and predictable security releases. Even so, we relied on external organizations to assign CVE IDs (Common Vulnerabilities and Exposures). This sometimes introduced administrative delays and extra coordination overhead.

Becoming a CNA (CVE Numbering Authority) allows the DSF to:

The initial exploration

The process began with internal discussions within the DSF Board and Django Security Team. We evaluated:

After confirming that our policies were mature and that the administrative workload would be manageable, we initiated the CNA application with MITRE.

Preparing the application

MITRE requires that new CNAs document their security processes and demonstrate that they can meet CNA obligations. Our preparation included:

  1. Reviewing and updating the Django Security Policy.

  2. Mapping our existing workflows to MITRE's CNA rules, including:

    1. How reports are received.
    2. How vulnerabilities are validated.
    3. How advisories are produced.
    4. How CVEs will be assigned and published.
  3. Defining the scope of the CNA:

    1. Django itself as the core product.
    2. A small, clearly bounded set of related ecosystem projects.
  4. Ensuring we had private communication channels and documented procedures for confidential handling.

  5. Drafting the required procedural documentation for MITRE.

Most of the work here was not about creating new processes but about articulating long standing Django practices in the format MITRE expects.

Training and review

Once our initial documentation was accepted, MITRE scheduled us for CNA onboarding training. This covered:

We also completed MITRE's required CNA onboarding exercises. As part of this process, we worked through sample security reports and demonstrated how we would determine CVE assignments, including cases where multiple CVEs may or may not be warranted for a single report.

Approval and onboarding

After MITRE approved our documentation, training, and exercise submissions, the DSF was formally granted CNA status. The announcements steps were:

Lessons learned

A few procedural insights for other projects considering CNA status:

What changes for Django users

For most contributors and users, nothing changes. Django will continue to follow its established process for receiving reports, coordinating fixes, and publishing security releases.

The difference is that the DSF can now assign CVE IDs directly, which simplifies coordination and allows us to publish advisories with fewer external dependencies.

Acknowledgments

This work was led by Django Fellows Natalia Bidart and Jacob Walls, with support from the Django Security Team and the DSF Board. We are grateful to MITRE for their guidance during the onboarding process.

If you have questions about Django's CNA scope or security process, contact the Django Security Team.

25 Jun 2026 11:00am GMT

24 Jun 2026

feedDjango community aggregator: Community blog posts

Supporting Django's Next Chapter

The path to hiring an Executive Director gained real momentum at DjangoCon US 2024, when Jacob Kaplan-Moss shared a vision for what dedicated resources could mean for the future of Django. In his blog post If We Had $1,000,000, he invited companies and supporters to help get the initiative off the ground. The response from the community was inspiring, and we're proud to see that vision become reality.

24 Jun 2026 7:00pm GMT

23 Jun 2026

feedPlanet Twisted

Glyph Lefkowitz: Adversarial Communication

As I have discussed in previous posts, "AIs" can make mistakes. In fact, they do make mistakes, and their mistake-making patterns are such that where and how they will make mistakes is both uncertain and constantly changing.

Thus, in any scenario where you want to attempt to make "productive" use of "AI", you must have a system in place for checking every result. Not checking some results; checking every result. If each result might have a consequence for you (and if it didn't have a consequence, why bother automating it?) and you cannot predict in advance which kinds of results will need verification, then verification is always required.

The verification often ends up being just as expensive as doing the work in the first place, which means that if you want your usage of "AI" to be personally profitable, you have to find someone else to externalize the cost of verification onto. This person becomes your adversary, and, if you are successful, your "AI's" victim.

The Ladder-Climber And Their Reverse-Centaur Rungs

One way that this constellation of facts can straightforwardly assemble themselves into a dystopian nightmare is the phenomenon, described by Cory Doctorow, of the reverse centaur. This is when your employer non-consensually turns you into the verification system. The "AI" does the fun part of initially performing the work, and then you do the boring part where you check if the robot is right and clean up its messes, even if everyone already knows that it would, in aggregate, be cheaper for you to do the work in the first place.

Reverse centaurs can be made from any automation, not only "AI" automation. I think that there is a reason that this term happens to have emerged in the "age of AI", though, and not with earlier automation technologies (even those which were considerably more viscerally horrific). That reason is: the wrongness of "AI" output is not merely a technical feature that must be compensated for, it is a generalized externality.

As I mentioned above, if you are responsible for the entirety of the work, both extruding the "AI" output and checking it, it's usually cheaper to have humans do the entirety of the work to begin with. When humans do the writing directly, we can check as we go, and thus verification doesn't need to be as comprehensive.

When "AI" coding advocates say "code review is the bottleneck", what they are observing is that the LLM is still rolling the dice for each PR, and a human is still necessary to verify that each of those rolls is a winner. But calling this process "code review" is a bit of a misnomer; it's not really "code review" in the traditional sense, it's human understanding.

Before the advent of "AI", the human understanding was implicit in the process of writing the code in the first place1, and the code review was a way of diffusing and extending that understanding. Now that the code can be authored with no initial understanding taking place, that cost has not gone away, it has moved.

Human understanding was always the bottleneck.

However, this is taking a collaborative view of a software project, where satisfying the needs and solving the problems of your customers are the goals. We can see that "AI" is a bad tool to satisfy those goals, because all it's doing is converting the first half of the work, that of understanding the code as you write it, to understanding the agent's output as you read it.

What if, instead, we were to take the view that every software company is a Hobbesian nightmare, red in tooth and claw? In this view, the only goal of a software project is for the individual developers to make their promo cycles and get their bonuses. Given that there is only a certain amount of money to go around, this is a zero-sum game where each programmer wants to look more productive than their colleagues.

Pretty much every organization finds it easy to reward "productivity" as expressed by lines of code emitted, but the benefits of doing thorough and thoughtful design, analysis, and code review very difficult to reward. In this world, an LLM is an invaluable tool for the sociopathic ladder-climber, particularly if your legacy organization is still structuring their workflows as if the person prompting the bot is "writing" the code, and then they get to foist off the act of "reviewing" the code onto someone else.

Here, the prompter effectively externalizes the cost of the LLM's failures but internalizes any benefits. The prompter will vibe-code a big feature, so large that the assigned reviewer can't possibly comprehend it all effectively. When this happens, the reviewer will, eventually, be pressured to approve it, even if they can try to spot a few problems along the way. The reviewer has their own work to get back to, after all, the obligation to review the prompter's (read: the bot's) code is a drain on their time that they are not going to get rewarded for.

If this feature is a big success, the prompter gets a promotion. If it causes a big issue, well, the reviewer must not have been careful enough.

This is why LLMs are "good for coding", and also why their biggest promoters keep having outages.

The Generative Gish Galloper

Coding is the biggest "success story" of this type of adversarial communication, but it is by far not the only instance of such a thing. LLMs create a new form of leverage that can turn Brandolini's law from a linear advantage into an exponential one. If you are engaged in a political debate where you want to overwhelm the other side in nonsense, an LLM can generate bullshit faster than it is physically possible for a human being to type, let alone respond thoughtfully. There is an asymmetry to the utility of this weapon as well: only one side of the political spectrum wants to flood the zone and destroy trust in institutions and the concept of truth. There's a good reason that the fascists love it.

Straightforward Spam and Fraud

This is kind of obvious, but LLMs can generate lightly-customized, plausible-looking text much more quickly than any human being. This facilitates their use in fraud, spam, and scams. In a spamming or fraudulent interaction, once again, the costs are externalized onto the victim: the recipient of a spam message has to do all the work of "checking" the LLM's output. Spammers already expect very low hit rates from boilerplate, and if the LLM can increase those percentages from 1% to 5% the technology will pay for itself; they don't need anything like reliable accuracy.

Customer "Support"

If you have any kind of commercial relationship with a company, I probably don't even need to mention this: customer "support" bots are a misery. Everybody knows it at this point. But customer support is usually conceptualized by businesses as an adversarial interaction, because it is a cost center. They maintain internal metrics on time-to-resolution and try to optimize them. Implicitly, this creates a dynamic where the goal of the customer service agent's job is not to solve your problem, but to emit noise that will cause you to think your problem is resolved, or to give up, as fast as possible. Unsurprisingly, LLMs can emit this noise faster than humans can, getting those customers off the phone. But those customers will remember those interactions, and the story outside the TTR metrics is horrible.

Similarly to the situation in software development, LLMs can look very good on paper for customer support, but mostly what they are doing is illuminating the problems with the industry's existing metrics, by turning "winning the metrics battle against the customer" into a more obvious and immediate defeat for the company's long term reputation.

"Education"

In 2026 it is sadly a fact of life that students cheat all the time using "AI", and that this cheating is very successful, in that the teachers find it very hard to detect.

LLMs are great for cheating on schoolwork because the student is externalizing the work of the checking onto the teachers, who are often starting at a disadvantage to begin with, at least in the US.

My view is that this is happening because of a divergence in the way that students vs. teachers (or, more accurately, "the broader educational system") view grading.

When a student is asked to write an essay, the teachers see the effort as both intrinsically worthwhile for the student, as well as useful as a pedagogical tool to evaluate and react to the student's progress. The student, by contrast, sees a stumbling block designed to knock them off the path to success and into a permanent underclass. It is no wonder that the student sees "AI" as useful to their own goals and has no compunction about deploying it.

There is a bitter irony that the ability to understand the inherent value of actually writing the essay on their own is the sort of thing that students can really only learn by writing a bunch of essays. There's no way that I can think of which makes the benefit legible as long as a shortcut is available.

The net effect here is a downward spiral, where the already-wobbling educational system is sustaining an attack that it doesn't have the resources to recover from. The individual students' attacks against their teachers and their schools' grading systems might appear to momentarily succeed, but they will win the battle and lose the war.

Spamming "For Good"?

Usually when we talk about someone unilaterally choosing to enter into an adversarial relationship, that's an "attack" and for good reasons we have a negative impression of the attacker. However, I would be remiss if I did not point out that there are some cases where the relationship was already adversarial; just because you're the attacker doesn't mean that you are evil.

For example we might imagine use-cases like automatically filing appeals for prior authorizations against health insurance. It's relatively well-known at this point that the main way for-profit insurers maintain their margins is by denying claims right up to the line of the policies themselves being fraud, so using a spamming tool to fight them might be entirely justifiable2 in that case.

Similarly, using an LLM could be justified in a fight against a company refusing to honor a warranty. One could imagine using an LLM to immediately generate replies and escalations.

However, even in imagined cases like these, the underlying problem is that the insurers and the vendors already have a tremendous amount of structural power, so it is more likely that they will have the advantage in deploying a communications weapon like an LLM, as well as enacting policies to simply ignore any LLM-based communication that you might submit. Worse, if these strategies were to become widespread, they might provide an excuse to reject any communications by feeding them into an unreliable "LLM detector" and issuing an automated "computer says no" even to hand-written correspondence.

It is also worth stressing that these cases are imagined, as compared to the very real coworker-abuse, spam, scam, fraud, and disinformation campaigns being waged in real life today.

Therefore, while legitimate uses might exist, it's hard to imagine that there's anywhere they would be genuinely valuable and sustainable. In the best case "AI" will provide a temporary advantage for underdogs that will provoke an arms race which the resource-advantaged adversaries will win in the long run, in the worst case the arms race itself will cement permanent structural change that will make things worse.

"Search" By Stealing

Most of the adversarial utility of "AI" is on the "write" side, since write-amplification is more obviously aggressive than reading. But the "read" side of LLMs - summarization and question-answering - can be a form of attack as well.

To begin with, the act of reading itself is currently enormously destructive, but that's arguably not a fundamental aspect of this technology. They could set reasonable rate-limits and respect things like robots.txt, as search engines have for decades now. They could also refrain from committing criminal levels of copyright infringement. But, today, using "AI" tools does suborn this sort of out-of-control crawling.

More insidiously, consider the scenario described in this YouTube video. The LTT Bros decided to try Linux again, and in the course of so doing, they had problems. When trying to solve these problems, they were faced with a choice: they could consult Reddit, or they could ask an LLM. Asking an LLM would "gaslight the heck out of" them, but they still found it preferable, because they would at least get an answer without getting yelled at.

Initially this sounds great. But it also means that you want to extract knowledge from a community, while mechanically eliding any values or norms that the community may want to impart as part of offering that knowledge. As someone who spent many years in a community tech support role, this is worrying. Many requests for support are people asking how to do things that will momentarily solve a superficial problem but create a long-term reliability problem or even an immediate security risk, that the question-asker doesn't want to hear about. Consider the question "I'm tired of entering my password so much, how do I make it so my laptop unlocks automatically". An obsequious chatbot will helpfully tell you how to do this without pushback.

But, this is also a sort of ethically murky area. The Linux community is somewhat famously, for many years now, a toxic cesspool of general hostility, misogyny, etc. It is certainly a good thing that people can get access to this knowledge without subjecting themselves to abuse. But it also means that the people with the power and the privilege to change the community for the better can just quietly withdraw, rather than fixing the problems. It also means that the positive elements of culture cannot be transmitted, and people will have no opportunity to learn about unknown unknowns.

In this case, the "adversarial" communication is with society. The thing that using an LLM for search lets you do is withdraw from society and avoid forming any personal connections. There are some personal connections which are painful and annoying, and so that can feel like a momentary balm. But the need to make connections in general is, like, the concept of society itself.

Who Am I Hurting?

LLMs are good at adversarial communication. They are so good at it, relative to their other benefits, that they will tend to make communications adversarial if you are not remaining vigilant about the possibility that it might do so. My request to you, dear reader, if you are going to use such tools, is to always ask yourself, "who might I be hurting, if I use an LLM for this?"

If you're using an "AI", who is its adversary? If you haven't given it one yet, who might the "AI" turn into an adversary? Who might you overwhelm with an asymmetric amount of output, or, if you're receiving information and not sending it, who are you taking that information from without consulting?

Figure out the answers to these questions and conduct yourself accordingly; the answer might be "yourself".

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor!


  1. One of the reasons that software developers tend to prefer greenfield development is that when you are given a blank page, you can project your own specific understanding onto it. You can structure the codebase in a way that works for your brain, down to the variable naming conventions and the module layouts. LLM-assisted development makes everything into instant brownfield work, which makes developers instantly miserable; even those who are excited about the technology will frequently complain about how it feels like their agency has been stolen and their joy in the work has been diminished. But I digress. ↩

  2. Modulo the massive amount of other externalities involved in using LLMs, of course, but I don't have the time or energy to get into those here. ↩

23 Jun 2026 8:06pm GMT

09 Jun 2026

feedPlanet Twisted

Hynek Schlawack: How to Ditch Codecov for Python Projects

Codecov's unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn't rely on third-party services.

09 Jun 2026 12:00am GMT

22 May 2026

feedPlanet Twisted

Glyph Lefkowitz: Opaque Types in Python

Let's say you're writing a Python library.

In this library, you have some collection of state that represents "options" or "configuration" for a bunch of operations. Such a set of options is a bundle of potentially ever-increasing complexity. Thus, you will want it to have an extremely minimal compatibility surface, with a very carefully chosen public interface, that is either small, or perhaps nothing at all. Such an object conveys state and might have some private behavior, but all you want consumers to be able to do is build it in very constrained, specific ways, and then pass it along as a parameter to your own APIs.

By way of example, imagine that you're wrapping a library that handles shipping physical packages.

There are a zillion ways to do it ship a package. There are different carriers who can ship it for you. There's air freight, and ground freight, and sea freight. There's overnight shipping. There's the option to require a signature. There's package tracking and certified mail. Suffice it to say, lots of stuff.

If you are starting out to implement such a library, you might need an object called something like ShippingOptions that encapsulates some of this. At the core of your library you might have a function like this:

1
2
3
4
5
async def shipPackage(
        how: ShippingOptions,
        where: Address,
    ) -> ShippingStatus:
    ...

If you are starting out implementing such a library, you know that you're going to get the initial implementation of ShippingOptions wrong; or, at the very least, if not "wrong", then "incomplete". You should not want to commit to an expansive public API with a ton of different attributes until you really understand the problem domain pretty well.

Yet, ShippingOptions is absolutely vital to the rest of your library. You'll need to construct it and pass it to various methods like estimateShippingCost and shipPackage. So you're not going to want a ton of complexity and churn as you evolve it to be more complex.

Worse yet, this object has to hold a ton of state. It's got attributes, maybe even quite complex internal attributes that relate to different shipping services.

Right now, today, you need to add something so you can have "no rush", "standard" and "expedited" options. You can't just put off implementing that indefinitely until you can come up with the perfect shape. What to do?

The tool you want here is the opaque data type design pattern. C is lousy with such things (FILE, pthread_*_t, fd_set, etc). A typedef in a header file can easily achieve this.

But in Python, if you expose a dataclass - or any class, really - even if you keep all your fields private, the constructor is still, inherently, public. You can make it raise an exception or something, but your type checker still won't help your users; it'll still look like it's a normal class.

Luckily, Python typing provides a tool for this: typing.NewType.

Let's review our requirements:

  1. We need a type that our client code can use in its type annotations; it needs to be public.
  2. They need to be able to consruct it somehow, even if they shouldn't be able to see its attributes or its internal constructor arguments.
  3. To express high-level things (like "ship fast") that should stay supported as we add more nuanced and complex configurations in the future (like "ship with the fastest possible option provided by the lowest-cost carrier that supports signature verification").

In order to solve these problems respectively, we will use:

  1. a public NewType, which gives us our public name...
  2. which wraps a private class with entirely private attributes, to give us an actual data structure, while not exposing the constructor,
  3. a set of public constructor functions, which returns our NewType.

When we put that all together, it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
from dataclasses import dataclass
from typing import Literal, NewType

@dataclass
class _RealShipOpts:
    _speed: Literal["fast", "normal", "slow"]

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("fast"))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("normal"))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("slow"))

As a snapshot in time, this is not all that interesting; we could have just exposed _RealShipOpts as a public class and saved ourselves some time. The fact that this exposes a constructor that takes a string is not a big deal for the present moment. For an initial quick and dirty implementation, we can just do checks like if options._speed == "fast" in our shipping and estimation code.

However, the main thing we are doing here is preserving our flexibility to evolve the related APIs into the future, so let's see how we might do that. For example, let's allow the shipping options to contain a concrete and specific carrier and freight method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
from dataclasses import dataclass
from enum import Enum, auto
from typing import NewType

class Carrier(Enum):
    FedEx = auto()
    USPS = auto()
    DHL = auto()
    UPS = auto()

class Conveyance(Enum):
    air = auto()
    truck = auto()
    train = auto()

@dataclass
class _RealShipOpts:
    _carrier: Carrier
    _freight: Conveyance

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.FedEx, Conveyance.air))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.UPS, Conveyance.truck))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.USPS, Conveyance.train))

def shippingDetailed(
    carrier: Carrier, conveyance: Conveyance
) -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(carrier, conveyance))

As a NewType, our public ShippingOptions type doesn't have a constructor. Since _RealShipOpts is private, and all its attributes are private, we can completely remove the old versions.

Anything within our shipping library can still access the private variables on ShippingOptions; as a NewType, it's the same type as its base at runtime, so it presents minimal1 overhead.

Clients outside our shipping library can still call all of our public constructors: shipFast, shipNormal, and shipSlow all still work with the same (as far as calling code knows) signature and behavior.

If you need to build and convey some state within your public API, while avoiding breakages associated with compatibility churn, hopefully this technique can help you do that!


Acknowledgments

Thanks for reading, and thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor.


  1. The overhead is minimal, but it is not completely zero. The suggested idiom for converting to a NewType is to call it like a function, as I've done in these examples, but if you are wanting to use this pattern inside of a hot loop, you can use # type: ignore[return-value] comments to avoid that small cost. ↩

22 May 2026 12:33am GMT