20 May 2025
Django community aggregator: Community blog posts
Python: a quick cProfile recipe with pstats
Python comes with two built-in profilers for measuring the performance of your code: cProfile and profile. They have the same API, but cProfile is a C extension, while profile is implemented in Python. You nearly always want to use cProfile, as it's faster and doesn't skew measurements as much.
By default, cProfile's CLI profiles a command and displays its profile statistics afterwards. But that can be a bit limited, especially for reading large profiles or re-sorting the same data in different ways.
For more flexibility, cProfile can instead save the profile data to a file, which you can then read with the pstats module. This is my preferred way of using it, and this post covers a recipe for doing so, with a worked example.
The recipe
First, profile your script:
$ python -m cProfile -o profile <script> [args]
Replace <script> with the path to your Python file, and [args] with any arguments you want to pass to it. cProfile will run your script under its profiling machinery, saving the results to a file called profile, as specified by the -o option.
Second, view the profile file using pstats:
$ python -m pstats profile <<< $'sort cumtime\nstats 1000' | less
The pstats CLI provides a REPL for interacting with profile files, based on its Stats class. The CLI is oddly undocumented, but its help command lists the available commands.
The above command passes several commands to pstats in a string. The string uses the $ syntax, a Bash feature for C-style strings, allowing \n to represent a newline, passing two commands:
sort cumtime: Sort the output by cumulative time, largest first. This means the time spent in a function and all its callees.stats 1000: Show the first 1,000 lines of the profile.
The output is passed to less, a common pager, allowing you to scroll through the results. Press q to quit when you're done!
Profile a module
If you're running a module instead of a script, add -m like:
$ python -m cProfile -o profile -m <module> [args]
Replace <module> with the name of the module you want to profile, and [args] with any arguments you want to pass to it.
Multiple profiles
If you're profiling code before and after, consider using different profile file names instead of just profile. For example, for checking the results of some optimization, I often use the names before.profile and after.profile, like:
$ python -m cProfile -o before.profile example.py
$ git switch optimize_all_the_things
$ python -m cProfile -o after.profile example.py
Alternative sort orders
To sort by other metrics, swap cumtime in sort cumtime for one of these values, per the Stats.sort_stats() documentation:
-
time: internal time-the time spent in the function itself, excluding calls to other functions.This is useful for finding the slowest functions in your code.
-
calls: number of calls to the function.This is useful for finding functions that are called many times and may be candidates for optimization, such as caching.
A Djangoey example
Here's a worked example showing how to apply this recipe to a Django management command. Say you are testing a database migration locally:
$ ./manage.py migrate example 0002
Operations to perform:
Target specific migration: 0002_complexito, from example
Running migrations:
Applying example.0002_complexito... OK
While it did pass, it was unexpectedly slow. To profile it, you would first reverse the migration to reset your test database:
$ ./manage.py migrate example 0001
...
Then you could apply the recipe to profile the migration.
First, stick the cProfile command in front of the migration command:
$ python -m cProfile -o profile ./manage.py migrate example 0002
Operations to perform:
Target specific migration: 0002_complexito, from example
Running migrations:
Applying example.0002_complexito... OK
Then, run the second pstats command to view the results:
$ python -m pstats profile <<< $'sort cumtime\nstats 1000' | less
This opens less with a long table, starting:
Welcome to the profile statistics browser.
profile% profile% Mon May 19 23:52:37 2025 profile
213287 function calls (206021 primitive calls) in 1.150 seconds
Ordered by: cumulative time
List reduced from 3576 to 1000 due to restriction <1000>
ncalls tottime percall cumtime percall filename:lineno(function)
425/1 0.001 0.000 1.150 1.150 {built-in method builtins.exec}
1 0.000 0.000 1.150 1.150 ./manage.py:1(<module>)
1 0.000 0.000 1.150 1.150 ./manage.py:7(main)
1 0.000 0.000 1.109 1.109 /.../django/core/management/__init__.py:439(execute_from_command_line)
...
The header tells us how many function calls were made, how many were primitive calls, and how long the code took to run. Then there's the table of all function calls, limited to 1,000 entries.
Since we're sorting by cumtime, cumulative time spent in each function, the first line shows the total time spent in all functions. That exec is cProfile running your code, and the later lines represent the top-level wrappers from Django.
Generally, it's best to find the first listed function within your code base. In this profile, you would search for ``example/ and find this entry:
ncalls tottime percall cumtime percall filename:lineno(function)
...
1 0.000 0.000 1.005 1.005 /.../example/migrations/0002_complexito.py:4(forward)
...
One call to the forward() function in the migration file took 1.005 seconds, nearly all of the 1.150 seconds total runtime. That's a bit suspicious!
Right above that entry, you might also spot the time spent running queries:
ncalls tottime percall cumtime percall filename:lineno(function)
...
13 0.000 0.000 1.007 0.077 /.../django/db/backends/utils.py:78(execute)
13 0.000 0.000 1.007 0.077 /.../django/db/backends/utils.py:88(_execute_with_wrappers)
13 0.000 0.000 1.007 0.077 /.../django/db/backends/utils.py:94(_execute)
13 0.000 0.000 1.007 0.077 /.../django/db/backends/sqlite3/base.py:354(execute)
13 1.006 0.077 1.006 0.077 {function SQLiteCursorWrapper.execute at 0x1054f7f60}
...
This stack of functions all show 13 calls, with a cumulative time of 1.007 or 1.006 seconds. They represent Django's database backend wrappers, which eventually pass the query to Python's SQLiteCursorWrapper.execute(), which is displayed differently because it's implemented in C.
So, we can tell that the migration ran 13 queries in total, and at least one of them was slow and ran by forward(). At this point, you might look at the source of forward() to see if you can find the slow query. But first, you might want to re-display the profile to show only the forward() function and its callees (the functions it called), which might shed some light on what it was doing.
To show only forward() and its callees, you can use the pstats callees command. This takes a regular expression to match the function names you want to show:
$ python -m pstats profile <<< $'sort cumtime\ncallees \\bforward\\b' | less
Welcome to the profile statistics browser.
profile% profile% Ordered by: cumulative time
List reduced from 3576 to 1 due to restriction <'\\bforward\\b'>
Function
called...
ncalls tottime cumtime
/.../example/migrations/0002_complexito.py:4(forward)
-> 1 0.000 0.000 /.../django/db/backends/utils.py:41(__enter__)
1 0.000 0.000 /.../django/db/backends/utils.py:44(__exit__)
1 0.000 1.005 /.../django/db/backends/utils.py:78(execute)
1 0.000 0.000 /.../django/utils/asyncio.py:15(inner)
1 0.000 0.000 {method 'create_function' of 'sqlite3.Connection' objects}
profile%
Goodbye.
(Output wrapped.)
This has revealed:
forward()only callsexecute()once, so there's only one slow query.- There's also a call to SQLite's
create_function(). It's fast, rounding down to 0.000 seconds, but perhaps may be something to do with the slow query.
Okay, time to look at the source:
def forward(apps, schema_editor):
import time
schema_editor.connection.connection.create_function(
"sleep",
1,
time.sleep,
)
with schema_editor.connection.cursor() as cursor:
cursor.execute("SELECT sleep(1)")
Ah, it's a deliberate pause that I added to show you this example. Well, that solves that mystery.
20 May 2025 4:00am GMT
16 May 2025
Django community aggregator: Community blog posts
Django News - Django News is at PyCon US this weekend! - May 16th 2025
Introduction
Django News is at PyCon US this weekend!
Jeff and Will are at PyCon US in Pittsburgh this weekend and would love to meet fellow Django enthusiasts. Drop by the DSF or JetBrains booth to say hello and connect with the many Django community members and DSF folks who will be around all weekend.
Django Newsletter
News
Google Summer of Code 2025 - Django Projects
Three projects out of many worth proposals were accepted. Improvements to Django admin, adding django-template-partials to core, and automating processes in the Django contribution workflow.
Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
Postgres 18 introduces asynchronous I/O with new io_method options (worker and io_uring), which can double or triple read performance in high-latency cloud environments.
Django Software Foundation
Simon Charette is the DSF member of the month
Simon Charette is a longtime Django contributor and community member. He served on the Django 5.x Steering Council and is part of the Security team and the Triage and Review team.
Updates to Django
Today 'Updates to Django' is presented by Abigail Afi Gbadago from the DSF Board and Djangonaut Space!🚀
Last week we had 10 pull requests merged into Django by 7 different contributors - including a first-time contributor! Congratulations to Safrone for having their first commits merged into Django - welcome on board!🎉
This week's Django highlights 🌟
- Security release of Django 5.2.1, 5.1.9 and 4.2.21.
- Field names have been added to hints in admin duplicated fields errors.
- Maximum bulk size for SQLite bulk_create and bulk_update methods now respect SQLITE_LIMIT_VARIABLE_NUMBER.
Django Newsletter
Wagtail CMS
Our four contributors for Google Summer of Code 2025
Four GSoC 2025 contributors will extend Wagtail with grid-aware sustainability, strict CSP compatibility, improved media listings, and enhanced keyboard shortcut accessibility.
Sponsored Link 1
Hire Django developers without the hassle!
Building a team of skilled Django developers has never been easier. Trust HackSoft to help you with strategic Django team augmentation. Learn more!
Articles
Django Security Best Practices: A Comprehensive Guide for Software Engineers
Enforce up-to-date Django versions, HTTPS, strong SECRET_KEY, ORM usage, built-in security middleware, XSS/CSRF defenses, robust authentication, dependency auditing, logging, and monitoring.
18 Years of REVSYS
Revsys marks 18 years offering Python and Django expertise, including code reviews, architectural design, cloud migrations, Kubernetes, CI/CD, AI integration, and team training.
Django: model field choices that can change without a database migration
Use Django 5.0 callable choices to avoid no-op migrations when updating model field choices, though database constraints still require migrations for data integrity.
Algorithms: Learning One's Learnings
Use Big O notation to choose efficient sorting in Django apps, leveraging Python's built-in Timsort or Quick Sort instead of Bubble Sort to improve performance.
Birds and Angles: Dabbling in Django Components
Combining django-bird and dj-angles enables Web-component style reusable Django template components for cleaner syntax and improved readability, despite limited filter parsing for props.
Setting up NGINX Unit (and switching from uWSGI)
Switch Django apps from uWSGI to NGINX Unit using JSON configuration, add SECURE_PROXY_SSL_HEADER, adjust socket proxy_pass, and enable ASGI/WSGI deployments.
My DjangoCon Europe 2025
Paolo Melchiorre recaps his DjangoCon Europe 2025 experience in Dublin through Mastodon posts covering keynotes, talks on testing, migrations, community events, and mentoring.
Tutorials
Rapid AI-powered applications with Django MongoDB and Voyage API
Learn how to build an LLM-powered recipe recommendation website with Django and MongoDB.
Podcasts
Django Chat #182: Event Sourcing with Chris May
Chris is a Senior Staff Engineer at WellSky, a software company in the health industry. We discuss his background as a graphic designer, learning Python (and Django) as an adult, his multiple conference talks on HTMX, why he's a fan of event sourcing, and more.
Talk Python #505: t-strings in Python (PEP 750)
A panel discussion of PEP 750 on t-strings, scheduled for Python 3.14, which build on the idea of f-strings to produce a template object rather than a standard string.
Django News Jobs
Python / Django Software Developer - full-time at Off Duty Management 🆕
Backend Python Developer (Django/DRF) at Paytree
Django Newsletter
Projects
astral-sh/ty
An extremely fast Python type checker and language server, written in Rust.
pydantic/pydantic-ai
Agent Framework / shim to use Pydantic with LLMs.
This RSS feed is published on https://django-news.com/. You can also subscribe via email.
16 May 2025 3:00pm GMT
My second DjangoCon Europe
Well I have been meaning to write this for over 2 weeks now, but better late than never! Towards the end of April 2025 I attended the DjangoCon Europe conference and Sprints and it was brilliant and exhausting all in one go.
Let's begin with the travel there, I decided to join those doing the SailRail for a relaxed train ride and crossing the sea to Dublin. This was great as I managed to make some use of the day (work and a blog post) while travelling as well as having some travel companions in the form of Thibaud, Sage, Tom & Daniele.
The next day kicked off the conference with an excellent keynote from Sarah Boyce, and other talks followed thoughout the next 2 days. Databases was a big theme along with community engagement and HTMX. However for me it was walking into the room and meeting folks from the community in person, that I have interacted with online for the past couple of years. This was also coupled with great conversations with friends new & old (mostly around making Django better). I also plucked up the courage and gave a lighting talk on the last day about my year of 100 words.
The evening socials again were excellent! Django Social on Wednesday and the official party on Friday, with a more chill evening going climbing with a couple of interested attendees. The weekend brought the Sprints which were just perfect. I managed to crack on with an open ticket/PR I have for the messages app in Django and also make some good progress on django-prodserver.
It was sad to leave, but reminds me that I want to go next year (if I am allowed by the family!). I am also excited with the energy I felt across the week reminding me that Django is going strong as ever and the communuity has a bright future. I could write more, but I am aware that I need to crack on with today's work, but I will leave you with the recommendation of getting to a DjangoCon if are use Django in any form, you will not be disappointed.
16 May 2025 5:00am GMT