fidzu : django

Features

There's a version of JsonResponse:

from django_msgspec.http import JsonResponse


def index(request):
    return JsonResponse({"title": "Hello, world!"})

…a test client with matching test case classes:

from django_msgspec.test import SimpleTestCase


class IndexTests(SimpleTestCase):
    def test_index(self):
        response = self.client.get("/", headers={"accept": "application/json"})
        assert response.status_code == 200
        # response.json() uses msgspec to parse the response body
        assert response.json() == {"title": "Hello, world!"}

…a version of Django's json_script template tag, which is where this whole story began:

{% load django_msgspec %}
{{ sales_by_product_id|json_script:"chart-data" }}

…and a handful of components that you activate purely through settings, with no code changes at all:

SESSION_SERIALIZER = "django_msgspec.sessions.JSONSerializer"

SERIALIZATION_MODULES = {
    "json": "django_msgspec.serializers.json",
    "jsonl": "django_msgspec.serializers.jsonl",
}

REST_FRAMEWORK = {
    "DEFAULT_RENDERER_CLASSES": ["django_msgspec.rest_framework.JSONRenderer"],
    "DEFAULT_PARSER_CLASSES": ["django_msgspec.rest_framework.JSONParser"],
}

That covers session storage and signing, dumpdata / loaddata in both JSON and JSON Lines, and DRF request parsing and response rendering.

Everything encodes with an enc_hook that knows about Django's lazy strings, so translated text passes through as you'd expect. It's all tested against the currently supported versions of Python and Django, with 100% coverage.

Déjà vu?

Only three weeks ago, I introduced django-orjson, a near-identical package backed by the Rust-powered orjson. So yeah, you might be confused why I made another faster-alternative-JSON-package-wrapper package so soon after the first one.

After releasing django-orjson, several folks from the community reached out to me telling me about the issues with orjson and pointing to msgspec instead. Additionally, while trying to roll out django-orjson on a client project, I learned that certain documented behaviours and limitations in orjson were going to be road blockers.

Here's a full list of what I learned:

• Dictionary keys have to be strings. Take a mapping of object ID to some count:

  >>> import json
  >>> json.dumps({1: "one"})
  '{"1": "one"}'
  
  >>> import orjson
  >>> orjson.dumps({1: "one"})
  Traceback (most recent call last):
    ...
  TypeError: Dict key must be str
  

  JSON objects can only have string keys, so the standard library coerces non-string keys with str(). But orjson refuses to do so, with the justification that the coercion is lossy and the keys will come back as strings when deserialized. Thankfully orjson does have an option here, OPT_NON_STR_KEYS, that opts in to the standard library behaviour, but you gotta know about it!

• Integers are limited to 64 bits.

  >>> json.dumps(2**64)
  '18446744073709551616'
  
  >>> orjson.dumps(2**64)
  Traceback (most recent call last):
    ...
  TypeError: Integer exceeds 64-bit range
  

  JSON itself sets no limit on number sizes, but RFC 8259 warns that implementations may, and many do-JavaScript, for one, silently loses precision beyond 2⁵³. orjson caps integers at 64 bits, matching native integer types, while Python's arbitrary-precision integers mean the standard library will happily emit larger values.

  In this case, orjson is probably more correct, but it is lacking an option to restore compatibility if required. (I didn't encounter any use case for massive numbers myself.)

• There's nowhere to report problems.

  The orjson README sayeth:

  There is no open issue tracker or pull requests due to signal-to-noise ratio.

  I have plenty of sympathy for that decision, having felt the weight of my own project's issue trackers backing up. But it does mean that when you hit any problems, you can't check whether it's known, follow a fix, or contribute one. Your options are to read the CHANGELOG and hope, or to work around it yourself.

• No sub-interpreter support, ever.

  The README also says:

  orjson does not and will not support PyPy, embedded Python builds for Android/iOS, or PEP 554 subinterpreters.

  I kinda missed this one when adopting orjson, but for me it's a bit of a concern. I think subinterpreters could support some cool use cases, like fast parallel test runners, and from my experience writing extension packages, support for them does not add much overhead.

  And this is not a soft limitation-you can't even import orjson in a sub-interpreter, let alone serialize anything:

  >>> from concurrent import interpreters
  >>> interp = interpreters.create()
  >>> interp.exec("import orjson")
  Traceback (most recent call last):
    ...
  concurrent.interpreters.ExecutionFailed: ImportError: module orjson.orjson does not support loading in subinterpreters
  

  It's a shame that the orjson maintainer advertises such a hard line here.

• No free-threading support yet.

  orjson currently publishes no wheels for free-threaded Python, which is now "stable" as of Python 3.14. On a free-threaded build, you're forced to compile the Rust package from source, which is a bit of an adoption blocker for large teams.

• Pydantic decided against it, on trust grounds.

  Back in 2019, a contributor opened a pull request to use orjson in Pydantic, and Samuel Colvin declined it. His stated reasons were:

  1. orjson's author gives no name or personal details on GitHub.
  2. The author had privately emailed Sam asking for the integration, and he was "surprised and somewhat worried by the hostile response I got when I made his/her request public".
  3. The compiled wheels for the package could potentially contain malicious code, which seems like more of a risk given the author's anonymity.

  He was careful to add: "Let me make it clear: I'm not accusing <the maintainer> of anything, I'm 99% certain that his/her intentions are honourable."

  I have the same feelings, now that I'm aware of all the (public) details of orjson. Pseudonymity is entirely legitimate and plenty of excellent software is written under a handle, and this was seven years ago. But stack it up with the closed issue tracker, and it does mean that installing orjson means trusting a compiled binary from a maintainer who has chosen not to engage in public at all.

msgspec's advantages

msgspec answers the questions raised by the above points against orjson:

• It handles numerical keys the way the standard library does:

  >>> import msgspec.json
  >>> msgspec.json.encode({1: "one"})
  b'{"1":"one"}'
  

• It encodes and decodes large integers:

  >>> import msgspec.json
  
  >>> msgspec.json.encode(2**64)
  b'18446744073709551616'
  
  >>> msgspec.json.decode(b"18446744073709551616")
  18446744073709551616
  

• It has an open issue tracker.

• msgspec fails to import in sub-interpreters right now, but the issue is being worked on by the maintainer and contributors.

• msgspec publishes free-threading compatible wheels today.

• The creator is not anonymous and the project is now maintained by a group in a GitHub organization, featuring at least Nikita Sobolev who I have known online from other open source projects (especially django-stubs).

And most importantly, msgspec's encoding and decoding are in the same performance ballpark as orjson's!

msgspec has standard library incompatibilities too

By the way, msgspec isn't fully compatible with json-here are the differences that I know about.

1. Non-finite floats are encoded as null rather than the standard library's NaN and Infinity:

>>> json.dumps(float("inf"))
'Infinity'

>>> msgspec.json.encode(float("inf"))
b'null'

I'd call this an improvement, since Infinity isn't valid JSON and other parsers will reject it. But it is a change, so if you rely on round-tripping those values, take note.

2. msgspec also only coerces keys that are string-like or number-like, so booleans and None are still rejected:

>>> json.dumps({None: "nothing"})
'{"null": "nothing"}'

>>> msgspec.json.encode({None: "nothing"})
Traceback (most recent call last):
  ...
TypeError: Only dicts with str-like or number-like keys are supported

Such keys should be rarer than numbers in practice.

What might come next in django-msgspec

django-msgspec covers the same ground as django-orjson today, but msgspec is a broader library than orjson, so there's more potential for future development.

First, msgspec's typed container class, Struct, lets you decode and validate in a single pass:

>>> import msgspec
>>> class Sale(msgspec.Struct):
...     product_id: int
...     count: int
...
>>> msgspec.json.decode(b'{"product_id": 1, "count": 2}', type=Sale)
Sale(product_id=1, count=2)

>>> msgspec.json.decode(b'{"product_id": "one", "count": 2}', type=Sale)
Traceback (most recent call last):
  ...
msgspec.ValidationError: Expected `int`, got `str` - at `$.product_id`

This could be useful for combining with Django views, DRF parsers, or even forms.

Second, msgspec can serialize and deserialize other data types, so they might be worth integrating.

Happy to take suggestions on the design here, on the issue tracker.

Fin

Please try out django-msgspec today and let me know how it goes.

May all your messages be to specification,

-Adam