01 May 2024

feedDjango community aggregator: Community blog posts

Self-Hosted Open Source - Michael Kennedy

Sponsor

01 May 2024 10:00pm GMT

29 Apr 2024

feedDjango community aggregator: Community blog posts

How to Use ModelAdmin with Wagtail CMS v6+

The powerful ModelAdmin feature of Wagtail CMS has been removed as of Wagtail v6. The Wagtail team now encourages the use of SnippetViewSet, which includes advanced features like bulk actions. Here is an official migration guide from ModelAdmin to Snippets.

However, if you have extensive custom code …

Read now

29 Apr 2024 11:04am GMT

Django: An admin extension to prevent state leaking between requests

Here's a small protection I added to a project a few years ago. I was considering it again since I saw a similar potential bug in a Django middleware.

Long live the ModelAdmin instances

Django's admin site is configured by the ModelAdmin class. You register this per model:

from django.contrib import admin

from example.models import Book


@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    fields = [
        "title",
        "special",
    ]

The @admin.register() decorator calls the AdminSite.register() method, which creates an instance of the ModelAdmin class (which can also be auto-generated):

class AdminSite:
    ...

    def register(self, model_or_iterable, admin_class=None, **options):
        ...
        for model in model_or_iterable:
            ...
            # Ignore the registration if the model has been
            # swapped out.
            if not model._meta.swapped:
                ...
                # Instantiate the admin class to save in the registry
                self._registry[model] = admin_class(model, self)

So, ModelAdmin instances are created once at import time and reused between all requests. That means it's not safe to use ModelAdmin instance variables to store state because they can affect later requests. (And if you run your project with a threaded WSGI server or ASGI server, ModelAdmin instance variables may be read by concurrent requests!)

This may be a surprising revelation if you have come from PHP, which has a "fresh process per request" model.

A leaky construction

In one project, I encountered a bug due to a developer misunderstanding this behaviour. Below is a simplified version of what the code did. Can you spot the bug?

from django.contrib import admin

from example.models import Book


@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    fields = [
        "title",
        "special",
    ]
    is_special_popup = False

    def changelist_view(self, request, extra_context=None):
        request.GET._mutable = True
        if request.GET.pop("special-popup", None):
            self.is_special_popup = True
        request.GET._mutable = False
        return super().changelist_view(request, extra_context=extra_context)

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        if self.is_special_popup:
            qs = qs.filter(special=True)
        return qs

This admin class's changelist could be opened in a popup for a particular process, which I've relabelled the "special popup". In that case, the QuerySet needed to be limited since only particular objects were relevant for selection.

changelist_view() pulls the query parameter from request.GET and stores it in is_special_popup. This "sneaky stashing" is needed because the default ModelAdmin.changelist_view() drops unknown query parameters before calling get_queryset().

The issue was that the is_special_popup variable would persist as True, once set, leaking into future requests. Later requests to a non-special-popup list would still be filtered.

This behaviour occurred because is_special_popup was set to False as a class-level and set to True as an instance-level variable. The accessing code, self.is_special_popup, reads either version, preferring the instance-level variable. Once a request set the instance-level variable, it was never cleared, leaving it around for future requests.

This issue went undiscovered because:

  1. In development, no one checked the main listing after opening the popup listing, at least without a runserver restart in between.
  2. In production, users either did not notice the limited results or the server processes always happened to restart between problematic requests.
  3. The unit tests used per-test admin class instances as a minor optimization over using Django's test client.

I discovered the issue when changing the tests to use setUpTestData() (a huge optimization I always like applying).

Plugging the leak

The fix I came up with was to store state on the request object:

@@ -7,19 +7,17 @@
 class BookAdmin(admin.ModelAdmin):
     fields = [
         "title",
         "special",
     ]
-    is_special_popup = False

     def changelist_view(self, request, extra_context=None):
         request.GET._mutable = True
-        if request.GET.pop("special-popup", None):
-            self.is_special_popup = True
+        request.is_special_popup = bool(request.GET.pop("special-popup", None))
         request.GET._mutable = False
         return super().changelist_view(request, extra_context=extra_context)

     def get_queryset(self, request):
         qs = super().get_queryset(request)
-        if self.is_special_popup:
+        if getattr(request, "is_special_popup", False):
             qs = qs.filter(special=True)
         return qs

Giving:

from django.contrib import admin

from example.models import Book


@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    fields = [
        "title",
        "special",
    ]

    def changelist_view(self, request, extra_context=None):
        request.GET._mutable = True
        request.is_special_popup = bool(request.GET.pop("special-popup", None))
        request.GET._mutable = False
        return super().changelist_view(request, extra_context=extra_context)

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        if getattr(request, "is_special_popup", False):
            qs = qs.filter(special=True)
        return qs

This version stores the "special popup" state on the request object, preventing inter-request leaking. Attaching extra attributes to request is the general pattern in Django, such as for request.session or request.site.

Permanent protection

After the initial fix, I wanted to ensure similar issues did not exist elsewhere, nor could they recur. I did this by moving the project to use custom AdminSite and ModelAdmin classes that block new attributes post-registration:

from functools import partial

from django.contrib import admin
from django.db.models.base import ModelBase


class ModelAdmin(admin.ModelAdmin):
    def __setattr__(self, name: str, value: bool | None) -> None:
        if getattr(self, "_prevent_attr_setting", False):
            clsname = self.__class__.__qualname__
            raise AttributeError(
                f"Cannot set attribute {name!r} on {clsname} after "
                + "registration. If you are trying to store per-request "
                + " attributes, they will leak between requests."
            )
        return super().__setattr__(name, value)


class AdminSite(admin.AdminSite):
    def register(self, model_or_iterable, admin_class=None, **options):
        if admin_class is None:
            raise TypeError("Must provide a ModelAdmin class")

        if not issubclass(admin_class, ModelAdmin):
            raise TypeError(f"Only subclasses of {__name__}.ModelAdmin may be used.")

        super().register(model_or_iterable, admin_class, **options)

        # Prevent further attributes from being set on the ModelAdmin class.
        # This cannot be done in ModelAdmin.__init__ because that prevents
        # subclasses from adding attributes.
        if isinstance(model_or_iterable, ModelBase):
            model_or_iterable = [model_or_iterable]
        for model in model_or_iterable:
            if model in self._registry:
                self._registry[model]._prevent_attr_setting = True


admin_site = AdminSite()

register = partial(admin.register, site=admin_site)

The custom ModelAdmin class wraps __setattr__() to conditionally block new attributes. This "lock" is enabled in the custom AdminSite.register() method. See Django's documentation for more on using a custom site.

This block worked well enough for me, and I believe it found another issue. But it isn't perfect-state may persist in other places, like module-level variables.

Should Django do something here?

This issue isn't specific to ModelAdmin. State can also persist between requests in other long-lived objects, at least in middleware classes. But class-based views are instantiated per request, at least (source).

I wonder if Django could add an "attribute block" to ModelAdmin, MiddlewareMixin, and maybe other places. It would require a deprecation path, but could flush out many latent and future bugs.

I have opened a forum discussion linked to this blog post. Please share your experiences and opinions there: "Add protection to some classes to prevent state leaking between requests?".

Fin

May your request bucket never leak,

-Adam

29 Apr 2024 4:00am GMT

26 Apr 2024

feedDjango community aggregator: Community blog posts

Django News - Djangonaut Space 2024 Session 2 - Apr 26th 2024

News

Djangonauts Space Session 2 Applications Open!

Applications are now open for the next 8-week group mentoring program, where individuals will work self-paced in a semi-structured learning environment.

If you want to work with Jeff on Django Packages, he is mentoring a team this session.

djangonaut.space

Django Developers Survey 2023 Results

The results of the Django Developers Survey 2023, which is a collaborative effort between the Django Software Foundation and PyCharm, are in!

jetbrains.com

10th anniversary of Django Girls Survey

Speaking of surveys, Django Girls and JetBrains have teamed up to celebrate the 10th anniversary of Django Girls with a survey that is complete with prizes.

jetbrains.com

Updates to Django

Today 'Updates to Django' is presented by Raffaella Suardini from Djangonaut Space!

Last week we had 9 pull requests merged into Django by 9 different contributors - including 3 first-time contributors! Congratulations to Mohammad Kazemi, Jason K Hall and Eyal Cherevatsky for having their first commits merged into Django - welcome on board!

Coming in Django 5.0.5, a bug fix with GeneratedField, preventing crashes when applying migrations, especially those involving the argument db_index=true.

Application for Djangonaut Space Open 🚀

I'm thrilled to announce that the second session of Djangonaut Space is open. You can read about past impressions and submit your application here.

Django Newsletter

Sponsored Ad

Free Trial of Scout APM Today!

Need answers to your Django app questions fast? Avoid the hassle of talking with a sales rep and the long wait times of large support teams, and choose Scout APM. Get Django insights in less than 4 minutes with Scout APM.

scoutapm.com

Articles

Announcing py2wasm: A Python to Wasm compiler · Blog · Wasmer

py2wasm converts your Python programs to WebAssembly, running them at 3x faster speeds

wasmer.io

How I organize `staticfiles` in my Django projects

Tips for configuring static files when using a framework like Tailwind.

joshthomas.dev

🐳 GitHub Actions Cleaning up old Docker container images

Jeff wrote up some notes on how to use GitHub Actions to clean up old container images.

webology.dev

Sending email in Django using GMail

Did you know you can use GMail for free, low-volume email sending?

softwarecrafts.co.uk

Django: Pinpoint upstream changes with Git

How to use Django's Git repository to search the commits between versions of Django rather than just relying on release notes.

adamj.eu

Events

DjangoCon US: Early-bird tickets now on sale!

DjangoCon US tickets are on sale. Early-bird individual tickets are $160 off, and early-bird corporate tickets are $100 off through May or until they sell out. Buy your ticket sooner and save a little before they are all gone!

Please note: DjangoCon US's CFP and Opportunity Grants have been extended through Monday, April 29th at 12 PM EDT. Check their website for updates.

djangocon.us

Django London Meetup May

Two talks at May's meetup hosted in the Kraken Tech London office.

meetup.com

Tutorials

Learn to use Websockets with Django by building your own ChatGPT

Everything you need to know about websockets to use them in your applications, with Django, channels, and HTMX.

saaspegasus.com

Videos

Django 2024: The Latest Development Trends

We'll take you through the latest Django Developers Survey results based on responses from 4,000 Django developers worldwide.

youtube.com

Sponsored Link

Boost Your Django DX, now updated for Django 5.0

Adam Johnson just updated this DX book with new content, a bunch of edits, and the latest versions of tools, including Python 3.12 and Django 5.0. Rated 5 stars with over 1,000 readers.

gumroad.com

Django News Jobs

Senior AI Engineer (f/m/d) at 1&1

Michigan Online Software Engineer at University of Michigan

Web developer at der Freitag Mediengesellschaft

Backend Software Architect, EarthRanger (Contract Opportunity) at AI2

Senior Software Engineer (backend) - IASO at Bluesquare

Remote Full-stack Python Developer at Scopic

Django Newsletter

Projects

django-allauth 0.62.0 released

With close to a dozen noteworthy changes, including adding support for logging in by email using a special code, also known as "Magic Code Login," Django-all auth 0.62.0 and 0.62.1 are worth checking out.

allauth.org

umami-software/umami

Umami is a simple, fast, privacy-focused alternative to Google Analytics.

github.com

danclaudiupop/robox

Simple library for exploring/scraping the web or testing a website you're developing.

github.com

Sponsorship

🌷 Spring Newsletter Sponsorships

Want to reach over 3,765+ active Django developers?

Full information is available on the sponsorship page.

django-news.com


This RSS feed is published on https://django-news.com/. You can also subscribe via email.

26 Apr 2024 3:00pm GMT

South: migraciones inversas

Una vez que pruebas las migraciones de bases de datos es muy complicado adaptarse a trabajar sin ellas. Cuando empezamos un proyecto Django uno de los primeros "requisitos" es instalar South para que haga todo este sucio trabajo.

Y no todo es bonito... normalmente estás obligado a ejecutar la migración para probar si funciona o no. Y si no funciona, sigues obligado a corregirla y volver a correr otra migración para arreglar la anterior que no funcionaba. Y este proceso puede repetirse varias veces a lo largo del desarrollo de un proyecto, con lo que al final acabas con un número muy alto de migraciones, de las cuales cierta cantidad son parches ínfimos para arreglar cualquier pequeño fallo.

Y si a todo esto le sumamos un equipo de trabajo, un repositorio de versiones y todo el flujo de trabajo que conlleva, puede ser una fuente importante de conflictos.

South schemamigration --auto --update

Imagino que a petición de la gente, en South han pensado en ello y han implementado una sencilla funcionalidad que resuelve todo este entuerto y hace más llevadero tanto el probar los cambios como la resolución de esos pequeños fallos que puedan surgir sin llenar de paja nuestro histórico. Se trata de la opción --update.

Para ponerlo en práctica podemos hacer una migración, cometer un error de sintaxis, arreglarlo y probar como la opción --update se encarga del resto de la magia.

$ ./manage.py schemamigration app --auto --update
 + Added model app.Group
Migration to be updated, 0026_auto__add_group, is already applied, rolling it back now...
previous_migration: 0025_auto__foo (applied: 2014-01-15 19:20:47)
Running migrations for app:
  - Migrating backwards to just after 0025_auto__foo.
  < partner:0026_auto__add_group
Updated 0026_auto__add_group.py. You can now apply this migration with: ./manage.py migrate app

Lo que este comando ha hecho es hacer backward de la última migración (la 0026) a la versión anterior 0025 y reescribir la migración 0026 con los nuevos cambios. Si todavía no fuera funcional o no nos convenciera, siempre podremos repetir el comando, quedándonos en la migración 0026, hasta que estemos convencidos de que todo está correcto; momento en el cual ya estaríamos capacitados para interactuar con el servidor de versiones correspondiente (commit/push).

Un poco más sobre las migraciones inversas

Esta tarde, trabajando con Borja, además del punto anterior se dió la circunstancia de que, habiendo partido del código actualizado de nuestro proyecto, ambos tuvimos que hacer una nueva migración en nuestros entornos de desarrollo; con lo que a la hora de actualizar los cambios, uno de los dos tendría que deshacer su migración para aceptar la del otro y luego la suya.

La última migración era la 0015, y ambos hicimos la 0016 en nuestros entornos de trabajo. En el momento de actualizar el código tomamos la decisión de que yo hiciera commit de los cambios y Borja se encargaba de arreglar el entuerto.

Pensamos que iba a ser más complicado, pero el proceso fue sencillo: migrar la aplicación hacia atrás a la versión 0015, eliminar la 0016 de disco, hacer pull de los cambios, aplicar la 0016 que había commiteado yo, hacer un schemamigration --auto para que generase la 0017 de Borja y, para finalizar, migrar la aplicación a la 0017, asegurarnos de que todo está correcto y, como paso final, enviar los cambios al servidor de versiones para yo poder aplicar también la 0017 en mi entorno de desarrollo:

$ ./manage.py migrate app 0015
$ rm app/migrations/0016_auto__add_whatever.py
$ darcs pull
$ ./manage.py migrate app 0016
$ ./manage.py schemamigration app --auto
$ ./manage.py migrate app 0017
$ darcs record

He recreado el proceso de memoria, (si se me olvida algo seguro que Borja hará buen uso de los comentarios). Personalmente pensé que sería más complejo que hacer un backward y eliminar un archivo (pensé que había que tocar la base de datos para eliminar el registro correspondiente de la tabla south_migrationhistory, pero se ve que no).

Y hasta aquí el capítulo de hoy, dos formas distintas de hacer migraciones inversas con South - con casuísticas distintas - sin morir en el intento. Ya ha merecido la pena el día.

26 Apr 2024 9:30am GMT

DjangoCMS, ventajas e inconvenientes

DjangoCMS es un proyecto que cada día se está volviendo más interesante. Se trata de un gestor de contenidos moderno con un montón de funcionalidades y características programado - obviamente - en Django. Hace menos de 1 mes, el 9 de abril para ser más exactos, ha salido la esperada versión 3.0 y, aunque todavía no la he probado, pinta bastante interesante.

Sí he trabajado en varios proyectos con la "antigua" versión 2.4.3 estable y tengo que decir que las sensaciones en general son bastante buenas, aunque como todo, con matices.

Es un proyecto relativamente sencillo de instalar y de configurar, cuenta con un montón de características atractivas como la edición live, una completa colección de plugins que permiten extender la funcionalidad del software para casi cualquier cosa y una sencilla personalización a través de placeholders en plantillas o la posibilidad de crear tus propios custom plugins de una manera fácil y rápida.

En el otro lado de la balanza tenemos que la documentación no es todo lo completa que debiera, a veces se pega con South, o que es bastante laborioso hacer una contribución al proyecto (curiosamente, una de las operaciones mejor documentadas).

En definitiva, si os apetece jugar un rato con DjangoCMS lo mejor es leerse el tutorial oficial y hacer alguna que otra prueba. Yo desde luego intentaré seguir de cerca la evolución de este proyecto open source y de sus hermanos gemelos que seguro darán mucho que hablar, Aldryn (de pago) y DjangoShop (e-commerce).

26 Apr 2024 9:30am GMT

Django: CACHE KEY_PREFIX

Una rápida para acabar el día, después de ponerme a leer la documentación de django.contrib.sitemaps y ver, por enésima vez, que reinventar la rueda es MAL, quise jugar otro poco con Memcached, porque tenía un issue bastante curioso.

Con instancias distintas del mismo software (Django) ejecutando varias páginas, todas contra Memcached, las caches se iban solapando de una forma muy divertida. Por ejemplo en la página index (/), la que primero cargaba y guardaba sus datos en memcache, sobreescribía a las demás y todas ofrecían el mismo contenido. Lo mismo con el resto de páginas con url coincidente.

Viendo el comportamiento la solución era sencilla, implementar KEY_PREFIX y que cada instsancia tuviera su propia "despensa de datos". Lo que en principio parecía trivial al final se convirtió en varias pruebas porque la documentación no está del todo clara (al menos para mí).

Después de varias pruebas, la configuración que ha funcionado adecuadamente ha sido la siguiente, en el archivo settings.py de cada instancia:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
        'KEY_PREFIX': 'your-custom-prefix_',
    }
}

Con cambiar la variable de configuración KEY_PREFIX dentro de la sección CACHES es suficiente, pero fuera de aquí no funcionará, como tampoco lo harán CACHE_PREFIX, CACHE_KEY_PREFIX ni CACHE_MIDDLEWARE_KEY_PREFIX, al menos no en mi entorno, por eso me hizo perder más tiempo del esperado. Así que ahí queda escrito, por si pudiera interesar.

26 Apr 2024 9:30am GMT

Django deploy: problems with limited hosting

Some months ago I had to deal with a Symfony2 project in a shared hosting (Spanish article) and now the big next deal is a similar task with Django.

The project is almost done and I have the hosting credentials, once I'm in I noticed that there is no chance to configure anything (Apache, WSGI or whatever) so I was a bit lost. Thanks to my Ailalelo's mates (they had lot of experience with this kind of situations) I found the proper configuration.

Hosting is django-ready, but the version they're running (1.4.2) is not the best choice, I want to install 1.6.x, the one I have used to develop the project. The other big requirement is virtualenv+pip to retrieve the packages I'm using.

Mainly I've solved it with two files, project.cgi and .htaccess.

project.cgi

The hosting structure is like many others, I have access to a homedir with the following directories:

myhome/
  logs/
  tmp/
  www/
    cgi-bin/
    index.html

Before to say Apache what to do with our project, let's install virtualenv and all the requirements, my choice is to put the environment out of the www/ directory:

myhome/
  env/
  logs/
  tmp/
  www/
    cgi-bin/
    project/
    index.html
$ virtualenv env
$ . env/bin/activate
$ pip install -r www/project/requirements/production.txt

Seems to be that apache's mod_cgi will process all the files you put in the cgi-bin directory, so we already know where to save our project.cgi. I have to tell apache to use my own virtualenv python, where the environment and the project are. And finally set some environment variables:

#!/home/myhome/env/bin/python

import os
from os.path import abspath, dirname
from sys import path

actual = os.path.dirname(__file__)
path.insert(0, os.path.join(actual, "..", "project/project"))
path.insert(0, os.path.join(actual, "..", "env"))

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['PATH'] = os.environ['PATH'] + ':/home/myhome/www/project/node_modules/less/bin'
os.environ['SECRET_KEY'] = 'SECRETKEY'
os.environ['DJANGO_SETTINGS_MODULE'] = "project.settings.production"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

Note that I modified the PATH because I have to be able to use less binary, required for django-compressor package. In this particular case my user was not allowed to install node/less in the system, so I had to install it locally, referencing the particular node_modules folder.

.htaccess

Now that Apache knows what to do, we should redirect almost all the incoming traffic to the cgi, so let's write some.htaccess rules:

AddHandler fcgid-script .fcgi
RewriteEngine On

RewriteRule ^static/(.*)$ project/project/static/$1 [L]
RewriteRule ^media/(.*)$ project/project/media/$1 [L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ cgi-bin/project.cgi/$1 [QSA,L]

AddDefaultCharset UTF-8

The media/ and static/ dirs are redirected to the proper location, because they didn't work as is. Not much more to say with this file, it's easy to understand I think.

Remember

26 Apr 2024 9:30am GMT

Django and memcache: clear cache keys

Let's play Django with Memcached. As the great framework Django is, it's so easy to activate any kind of cache in your project. Memcached is one of the options, but you can also work with DatabaseCache, FileBasedCache, LocMemCache, MemcachedCache, DummyCache (a kind of non-cache very useful for devel/test enviroments) or - of course - your own CustomCache if you want.

Activating cache

It's too easy to activate the cache feature, it's enough to set the preferences in settings, install python-memcached in your enviroment (in case you will use MemcachedCache), and not much more to do. A couple of examples:

1. Basic FileBasedCache settings:

# settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
        'LOCATION': '/var/tmp/django_cache',
    }
}

2. MemcachedCache settings:

# settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

3. Depending on the enviroment you can use MemcachedCache and DummyCache:

# settings.devel.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.dummy.DummyCache',
    }
}
# settings.production.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

Setting the places where cache will act

Now that we have our project with a configured kind of cache, we must to say where and when to activate it. There are multiple ways to do it (on the whole site, views, templates, urls...). I'm going to use... let's say urls. So, in our urls.py we have to set a time and activate the cache:

# urls.py

from django.views.decorators.cache import cache_page

ctime = (60 * 24) # A day
urlpatterns = patterns('',
    url(r'^$',
        cache_page(ctime)(BlogIndexView.as_view()),
        {},
        'blog-index'
        ),
        ...
)

A simple server reload will be enough to have cache running. We can see it on action with django-debug-toolbar, django-memcache-status or something like that.

Clean a specific key cache

And now the funniest part. For example, talking about a blog tool, when you write a new post (or editing older one) the software should be able to remove some cache keys, i.e. the blog-index one (because you have a new post) and the post-detail other (because you must be able to inmediately see the changes in the post you're editting).

Following this link I've created a cache.py with this content:

# cache.py

# -*- coding: utf-8 -*-

def expire_view_cache(
    view_name,
    args=[],
    namespace=None,
    key_prefix=None,
    method="GET"):

    """
    This function allows you to invalidate any view-level cache.

    view_name: view function you wish to invalidate or it's named url pattern
    args: any arguments passed to the view function
    namepace: optioal, if an application namespace is needed
    key prefix: for the @cache_page decorator for the function (if any)

    http://stackoverflow.com/questions/2268417/expire-a-view-cache-in-django
    added: method to request to get the key generating properly
    """

    from django.core.urlresolvers import reverse
    from django.http import HttpRequest
    from django.utils.cache import get_cache_key
    from django.core.cache import cache
    from django.conf import settings
    # create a fake request object
    request = HttpRequest()
    request.method = method
    if settings.USE_I18N:
        request.LANGUAGE_CODE = settings.LANGUAGE_CODE
    # Loookup the request path:
    if namespace:
        view_name = namespace + ":" + view_name
    request.path = reverse(view_name, args=args)
    # get cache key, expire if the cached item exists:
    key = get_cache_key(request, key_prefix=key_prefix)
    if key:
        if cache.get(key):
            cache.set(key, None, 0)
        return True
    return False

And last step is to call the expire_view_cache function on model form save hook (admin.py in this case):

# admin.py

from cache import expire_view_cache

class PostAdminForm(admin.ModelAdmin):
    ...
    def save_model(self, request, obj, form, change):
        expire_view_cache("blog-index")
        expire_view_cache("post-detail", [obj.slug])

And that's all, we are able to clean/purge/remove the cache when a new post is added or edited. As you can see in the code, cache is fun but you have to be careful to set it on the right way.

26 Apr 2024 9:30am GMT

Python: Diffing unit tests to keep a copy-pasted code in sync

Copy-paste-tweaking library code feels like a dirty but inevitable programming practice. Often driven by deadlines or other constraints, it seems all projects end up with something copy-pasted in and tweaked for one specific use case.

When we find ourselves doing this, it's essential to consider the long-term maintenance of those copies. After all, "software engineering is programming integrated over time" (see previously). We want to add a defence that alerts us to any relevant upstream changes. But since that is hard to do robustly, it is often omitted.

One approach is to maintain a fork, but that is heavy-handed and requires per-release maintenance. In this post, we'll cover an alternative I recently tried, using a unit test. This test asserts that the diff between the upstream code and our project's copy-pasted version is constant. The test fails if either version changes, smoothing upgrades and ensuring we consider any further tweaks.

A Djangoey example

I recently worked on a Django project that heavily extends Django's admin site. Most of these extensions were done as usual, extending classes or templates as required. However, one case needed a copy-paste-tweak of the upstream "fieldset" template used to render form fields. That tweak looks something like this:

                 {% if field.is_readonly %}
                     <div class="readonly">{{ field.contents }}</div>
                 {% else %}
-                    {{ field.field }}
+                    {% block field %}
+                        {{ field.field }}
+                    {% endblock %}
                 {% endif %}
             {% endif %}
         </div>

The extra {% block %} allows extending templates to modify the rendering of select fields.

When upgrading to a later Django version, the upstream template and corresponding CSS changed. That caused the tweaked template to render incorrectly since it still had the old base. In particular, the fields stopped stacking horizontally, leading to some unusably lengthy pages.

The fix was to integrate the upstream changes into the copied template. Doing so revealed that some smaller changes had also been missed from previous Django versions. I added a diffing unit test like the one below to ensure future upstream changes will not be missed.

import difflib
import re
from pathlib import Path
from textwrap import dedent

import django
from django.conf import settings
from django.test import SimpleTestCase


class CopiedTemplateTests(SimpleTestCase):
    """
    Tests to check synchronization of templates that we've copy-paste-tweaked
    from Django. These tests fail when either version changes, so we may need
    to integrate upstream changes before regenerating the included diffs.

    Get updated diffs on failure by using pytest --pdb and print(diff).
    """

    def test_admin_includes_fieldset(self):
        upstream_version = (
            (
                Path(django.__path__[0])
                / "contrib/admin/templates/admin/includes/fieldset.html"
            )
            .open()
            .readlines()
        )
        our_version = (
            (settings.BASE_DIR / "templates/admin/includes/fieldset.html")
            .open()
            .readlines()
        )
        diff = "".join(
            difflib.unified_diff(
                upstream_version, our_version, fromfile="upstream", tofile="ours"
            )
        )
        diff = re.sub(r"^ \n", "\n", diff, flags=re.MULTILINE)
        expected_diff = dedent(
            """\
            --- upstream
            +++ ours
            @@ -17,7 +17,9 @@
                                             {% if field.is_readonly %}
                                                 <div class="readonly">{{ field.contents }}</div>
                                             {% else %}
            -                                    {{ field.field }}
            +                                    {% block field %}
            +                                        {{ field.field }}
            +                                    {% endblock %}
                                             {% endif %}
                                         {% endif %}
                                     </div>
            """
        )
        assert diff == expected_diff

Here's how the test works:

  • The two template files are read into lists of lines using pathlib. The path for the upstream version is computed from the Django module's __path__ attribute, which will be inside the project's virtual environment. The project version uses Django's BASE_DIR setting, which points at the project root.
  • The diff between the two versions is computed using Python's difflib.unified_diff(). It's neat this is built-in!
  • The diff is modified with a regular expression to strip the whitespace on blank lines. This is to make it compatible with the expected diff.
  • The diff is compared with its expected version. To keep the expected diff inside the test without weird indentation, its multiline string is dedented with textwrap.dedent().

When the test fails, under pytest, it looks like this:

>       assert diff == expected_diff
E       AssertionError: assert '--- upstream...     </div>\n' == '--- upstream...     </div>\n'
E
E         Skipping 326 identical leading characters in diff, use -v to show
E         - lock fields %}
E         ?           -
E         + lock field %}
E           +                                        {{ field.field }}
E           +                                    {% endblock %}...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

This "diff of diffs" isn't the easiest to read, but it at least gives an idea of where the unexpected differences lie. Unfortunately, the failure can't differentiate whether the upstream or project version changed, but that should be obvious in most situations.

Per the docstring, the updated diff can be retrieved by running pytest with its --pdb option and print(diff):

$ pytest --pdb example/tests.py
========================= test session starts =========================
...
example/tests.py:55: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>
> /.../example/tests.py(55)test_admin_includes_fieldset()
-> assert diff == expected_diff
(Pdb) print(diff)
--- upstream
+++ ours
@@ -17,7 +17,9 @@
                                 {% if field.is_readonly %}
                                     <div class="readonly">{{ field.contents }}</div>
                                 {% else %}
-                                    {{ field.field }}
+                                    {% block field %}
+                                        {{ field.field }}
+                                    {% endblock %}
                                 {% endif %}
                             {% endif %}
                         </div>

(Pdb)

This can then be copy-pasted back into the test file.

With this test in place, I am confident that the project will merge future upstream changes to this template.

Diffing classes and functions

This approach can be adapted to copy-paste-tweaked classes or functions by using Python's inspect module to gather their source code. Whilst I'd normally recommend subclassing, or patching with patchy, it could be helpful when edits to the middle of a function are required. Below is an imagined example with a modified copy of Django's timesince filter.

import difflib
import inspect
import re
from textwrap import dedent

from django.test import SimpleTestCase
from django.utils.timesince import timesince as upstream_timesince

from example.timesince import timesince as our_timesince


class CopiedFunctionTests(SimpleTestCase):
    """
    Tests to check synchronization of functions that we've copy-paste-tweaked.
    These tests fail when either version changes, so we may need to integrate
    upstream changes before regenerating the included diffs.

    Get updated diffs on failure by using pytest --pdb and print(diff).
    """

    def test_timesince(self):
        upstream_version = inspect.getsource(upstream_timesince).splitlines(
            keepends=True
        )
        our_version = inspect.getsource(our_timesince).splitlines(keepends=True)
        diff = "".join(
            difflib.unified_diff(
                upstream_version, our_version, fromfile="upstream", tofile="ours"
            )
        )
        diff = re.sub(r"^ \n", "\n", diff, flags=re.MULTILINE)
        expected_diff = dedent(
            """\
            --- upstream
            +++ ours
            @@ -45,6 +45,10 @@
                 if reversed:
                     d, now = now, d
                 delta = now - d
            +
            +    # Return "Now" for small differences.
            +    if -10 <= delta.total_seconds() <= 10:
            +        return "Now"

                 # Ignore microseconds.
                 since = delta.days * 24 * 60 * 60 + delta.seconds
            """
        )
        assert diff == expected_diff

This test works similarly to the one before. The difference is that each function's source code is retrieved using inspect.getsource().

Fin

Let me know if you try this technique and how well it works.

Never split the difference,

-Adam

26 Apr 2024 4:00am GMT

25 Apr 2024

feedDjango community aggregator: Community blog posts

Python: Make line number paths with inspect

Many terminals and text editors support what I'll call "line number paths" of the form <filename>:<lineno>. Opening that path, whether by clicking or passing to a CLI, opens the given file at that line.

Python's inspect module has a couple of functions that can be combined to make such paths, for a given class or function. Here's the recipe:

from inspect import getsourcefile, getsourcelines

print(f"{getsourcefile(obj)}:{getsourcelines(obj)[1]}")

getsourcefile() returns the file the object is defined in. getsourcelines() returns a tuple, containing the list of source code lines and the first line number, hence the [1] to select just the line number.

For example, to make a path for a function in Django:

In [1]: from django.utils.html import format_html

In [2]: from inspect import getsourcefile, getsourcelines
   ...: print(f"{getsourcefile(obj)}:{getsourcelines(obj)[1]}")
/.../.venv/site-packages/django/utils/html.py:95

I have found this recipe handy a couple of times for batch edits. For example, I recently upgraded django-import-export on a project. Due to an upstream change, I needed to check every subclass of django-import-export's ModelResource class in the project. I could have searched the code for all such classes, but that can be complicated due to inheritance. Instead, I used class.__subclasses__() to find all subclasses and made "line number paths" for each:

In [1]: from inspect import getsourcefile, getsourcelines

In [2]: from import_export.resources import ModelResource

In [3]: for cls in ModelResource.__subclasses__():
   ...:     init = cls.__init__
   ...:     if init is not ModelResource.__init__:
   ...:         print(f"{getsourcefile(init)}:{getsourcelines(init)[1]}")
   ...:
/.../example/apples/resources.py:1136
/.../example/apples/resources.py:1239
/.../example/bananas/resources.py:351
/.../example/bananas/resources.py:502
/.../example/bananas/resources.py:1874
/.../example/cherries/resources.py:297
...

I was then free to edit each link in turn.

Fin

May you find more creative ways to use Python's inspection features,

-Adam

25 Apr 2024 4:00am GMT

24 Apr 2024

feedDjango community aggregator: Community blog posts

Workbench, the Django-based agency software

Workbench, the Django-based agency software

I get the impression that there's a lot of interesting but unknown software in Django land. I don't know if there's any interest in some of the packages I have been working on; if not this blog post is for myself only.

(Hi)story time

As people may know I work at Feinheit, an agency which specializes in digital communication services for SMEs, campaigns for referendums, and website and webapp development. At the time of writing we are a team of about 20-25 communication experts, graphic designers, programmers and project managers.

We have many different clients and are working on many different projects at the same time and are billing by the hour1. Last year my own work has been billed to more than 50 different customers. In the early days we used a shared file server with spreadsheet files to track our working hours. Luckily we didn't often overwrite the edits others made but that was definitely something which happened from time to time.

We knew of another agency who had the same problems and used a FileMaker-based software. Their solution had several problems, among them the fact that it became hard to evolve and that it got slower and slower as more and more data was entered into it over the years. They had the accounting know how and we had the software engineering know how so we wrote a webapp based on the Django framework. As always, it was much more work than the initial estimate, but if we as programmers didn't underestimate the effort needed we wouldn't have started many of the great projects we're now getting much value and/or enjoyment from, hopefully both. The product of that work was Metronom. The first release happened a little bit later than harvest but it already came with full time tracking including an annual working time calculator, absence management, offers, invoices including PDF generation etc, so it was quite a bit more versatile while still being easier to use than "real" business software solutions.

I personally was of the opinion that the product was good enough to try selling it, but for a variety of reasons (which I don't want to go into here) this never happened and we decided that we didn't want to be involved anymore.

However, this meant that we were dead-end street with a software that didn't belong to us anymore, which wasn't evolving to our changing requirements. I also didn't enjoy working on it anymore. Over the years I have tried replacing it several times but that never came to pass until some time after the introduction of Holacracy at our company. I noticed that I didn't have to persuade everyone but that I, as the responsible person for this particular decision, could "just"2 move ahead with a broad interpretation of the purpose and accountabilities of one of my roles.

Workbench

Screenshot

Workbench is the product of a few long nights of hacking. The project was started as an experiment in 2015 and was used for sending invoices but that wasn't really the intended purpose. After long periods of lying dormant I have brought the project to a good enough state and switched Feinheit away from Metronom in 2019.

I have thought long and hard about switching to one of the off-the-shelf products and it could very well be that one of them would work well for us. Also, we wouldn't have to pay (in form of working hours) for the maintenance and for enhancements ourselves. On the other hand, we can use a tool which is tailored to our needs. Is it worth the effort? That's always hard to answer. The tool certainly works well for the few companies which are using it right now, so there's no reason to agonize over that.

At the time of writing, Workbench offers projects and services, offers and invoices incl. PDF generation, recurring invoices, an address book, a logbook for rendered services, annual working time reports, an acquisition funnel, a stupid project planning and resource management tool and various reports. It has not only replaced Metronom but also a selection of SaaS (for example Pipedrive and TeamGantt) we were using previously.

The whole thing is open source because I don't want to try making agency software into a business anymore, I only want to solve the problems we have. That being said, if someone else finds it useful then that's certainly alright as well.

The license has been MIT for the first few years but I have switched to the GPL because I wanted to integrate the excellent qrbill module which is also licensed under the GPL. As I wrote elsewhere, I have released almost everything under the GPL in my first few open source years but have switched to BSD/MIT later when starting to work mainly with Python and Django because I thought that this license is a better fit3 for the ecosystem. That being said, for a product such as this the GPL is certainly an excellent fit.

Final words?

There are a few things I could write about to make this into a series. I'm putting a few ideas here, not as an announcement, just as a reminder for myself.


  1. Rather, in six minute increments. It's even worse.

  2. Of course it's never that simple, because the responsibility is a lot. The important point is: It would have been a lot of work anyways, the big difference is that it was sufficient to get consent from people; no consensus required.

  3. That's not meant as a criticism in any way!

24 Apr 2024 5:00pm GMT

Django: Pinpoint upstream changes with Git

Django's release notes are extensive and describe nearly all changes. Still, when upgrading between Django versions, you may encounter behaviour changes that are hard to relate to any particular release note.

To understand whether a change is expected or a regression, you can use Django's Git repository to search the commits between versions. I often do this when upgrading client projects, at least the larger ones.

In this post, we'll cover Django's branching structure, determining and searching through those commits, a worked example, and advanced behavioural searching with git bisect.

Django's branching structure

Most open source projects use a single main branch, tagged for release when appropriate. However, because Django maintains support for multiple feature versions simultaneously, its branching structure is more complicated. Here's an example:

* main
|
| * stable/5.0.x
| |
⋮ ⋮
|/
|
*
⋮
* * stable/4.2.x
| |
⋮ ⋮
|/
|
*
⋮

There's a main branch, representing the future version of Django, and stable/<version>.x branches representing released versions (at least, released in alpha). When it is time for an alpha release of a new version, a new stable/<version>.x branch is created from main.

Commits are always merged to main. Then, they may be copied onto relevant stable/<version>.x branches with git cherry-pick, also known as backporting, if the merger deems relevant (mergers are typically the Django fellows). Typically, only bug fixes are backported, depending on Django's Supported Versions Policy.

(If you're particularly interested, the backporting script is hosted in Django's wiki.)

Clone and update Django's repository

Before inspecting inter-version history, ensure you have an up-to-date clone of Django's repository. If you're cloning fresh, you're fine. But if you have an existing clone, you want to update any local stable/* branches so they include all backported commits. Here's a short command using git for-each-ref and a while loop to run git pull on all local stable/ branches:

$ git for-each-ref 'refs/heads/stable/*' --format="%(refname:short)" | \
while read entry
do
  git switch $entry
  git pull
done

For example, here's what I see when I run it when all branches are up-to-date:

Switched to branch 'stable/3.0.x'
Your branch is up to date with 'upstream/stable/3.0.x'.
Already up to date.
Switched to branch 'stable/3.1.x'
Your branch is up to date with 'upstream/stable/3.1.x'.
Already up to date.
...
Switched to branch 'stable/5.0.x'
Your branch is up to date with 'upstream/stable/5.0.x'.
Already up to date.

Find changes between versions

To see the changes between versions <old> and <new>, we want commits starting at the point <old> branched from main and ending at the tip of the <new> branch. This start point is what Git calls the merge base between <old> and main, as it's the base that a merge between <old> and main would use for conflicts. Git's git merge-base command can report the merge base between two branches. For example, the point that Django 4.2 branched from main is:

$ git merge-base main stable/4.2.x
9409312eef72d1263dae4b0303523260a54010c5

We can double-check with git show:

$ git show --stat 9409312eef72d1263dae4b0303523260a54010c5
commit 9409312eef72d1263dae4b0303523260a54010c5
Author: Mariusz Felisiak <...>
Date:   Sun Jan 15 19:12:57 2023 +0100

    Updated man page for Django 4.2 alpha.

 docs/man/django-admin.1 | 434 ++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------------------------------------------------------
 1 file changed, 142 insertions(+), 292 deletions(-)

Yup, that looks right-it's ex-Fellow Mariusz preparing for the Django 4.2 alpha release right before creating stable/4.2.x.

For the complete log between <old> and <new>, use git log with git merge-base to template the start point. Use the two-dot range syntax to select commits between two points. So, from Django 4.2 to 5.0:

$ git log --oneline $(git merge-base main stable/4.2.x)..stable/5.0.x

For the diff between versions, use git diff with its three-dot syntax between the versions. Note that Git is inconsistent here: three dots for diff mean the same as two dots for log.

$ git diff $(git merge-base main stable/4.2.x)...stable/5.0.x

Both commands give an overwhelming number of changes (go Django contributors!). They get more useful when you add some filtering options. For example, use git log -S to narrow commits to those that added or removed a given string:

$ git log -S FORMS_URLFIELD_ASSUME_HTTPS $(git merge-base main stable/4.2.x)...stable/5.0.x
commit 92af3d4d235448446e53e982275315bedcc4c204
Author: Mariusz Felisiak <...>
Date:   Tue Nov 28 20:04:21 2023 +0100

    [5.0.x] Refs #34380 -- Added FORMS_URLFIELD_ASSUME_HTTPS transitional setting.

    This allows early adoption of the new default "https".

    Backport of a4931cd75a1780923b02e43475ba5447df3adb31 from main.

Or use git diff with a pathspec to limit the diff to particular files:

$ git diff $(git merge-base main stable/4.2.x)...stable/5.0.x -- 'django/contrib/admin/*.css'
diff --git django/contrib/admin/static/admin/css/base.css django/contrib/admin/static/admin/css/base.css
index 72f4ae169b3..44f2fc8802e 100644
--- django/contrib/admin/static/admin/css/base.css
+++ django/contrib/admin/static/admin/css/base.css
@@ -22,11 +22,11 @@ :root {

     --breadcrumbs-fg: #c4dce8;
     --breadcrumbs-link-fg: var(--body-bg);
-    --breadcrumbs-bg: var(--primary);
+    --breadcrumbs-bg: #264b5d;
...

There are many other useful git log and git diff options for narrowing down changes. I cover my top picks in Boost Your Git DX.

A worked example

I've recently been working on upgrading a client project from Django 4.1 to 4.2. One change that I found was an admin unit test started failing its assertNumQueries() assertion. The test looked like this:

class BookAdminTest(TestCase):
    ...

    def test_save_change_list_view_num_of_queries(self) -> None:
        ...  # Some setup

        with self.assertNumQueries(7):
            """
            1. SAVEPOINT ...
            2. SELECT "django_session" ...
            3. SELECT "core_user" ...
            4. SELECT COUNT(*) AS "__count" FROM "library_book"
            5. SELECT COUNT(*) AS "__count" FROM "library_book"
            6. SELECT "library_book" ...
            7. RELEASE SAVEPOINT ...
            """
            self.client.post("/admin/library/book/", ...)

And the failure message looked like this:

E   AssertionError: 9 != 7 : 9 queries executed, 7 expected
E   Captured queries were:
E   1. SAVEPOINT ...
E   2. SELECT "django_session" ...
E   4. SELECT COUNT(*) AS "__count" FROM "entity_entity"
E   5. SELECT COUNT(*) AS "__count" FROM "entity_entity"
E   6. SELECT "entity_entity" ...
E   7. SAVEPOINT ...
E   8. RELEASE SAVEPOINT ...
E   9. RELEASE SAVEPOINT ...

There were two extra queries. Thankfully, the test author had diligently copied simplified versions of the queries into a comment. I could easily compare and see the new queries were #7, a SAVEPOINT, and #8, a RELEASE SAVEPOINT. These are SQL for a nested transaction, and in Django we typically use transaction.atomic() to create transactions, nested or not.

I didn't immediately spot any relevant release note for this extra transaction, so I checked Django's history. I checked Django's Git log for commits that:

  1. Were between version 4.1 and 4.2.
  2. Added or removed the string "atomic", with git log -S.
  3. Affected django/contrib/admin, with pathspec limiting.

The combined command found precisely the responsible commit straight away:

$ git log $(git merge-base main stable/4.1.x)..stable/4.2.x -S atomic -- django/contrib/admin
commit 7a39a691e1e3fe13588c8885a222eaa6a4648d01
Author: Shubh1815 <...>
Date:   Sat Sep 24 15:42:28 2022 +0530

    Fixed #32603 -- Made ModelAdmin.list_editable use transactions.

Looking at the commit, it turned out it added a pretty clear release note:

$ git show 7a39a691e1e3fe13588c8885a222eaa6a4648d01
commit 7a39a691e1e3fe13588c8885a222eaa6a4648d01
Author: Shubh1815 <shubhparmar14@gmail.com>
Date:   Sat Sep 24 15:42:28 2022 +0530

    Fixed #32603 -- Made ModelAdmin.list_editable use transactions.

...
diff --git docs/releases/4.2.txt docs/releases/4.2.txt
index 5a849cbbe5..5774bfef7b 100644
--- docs/releases/4.2.txt
+++ docs/releases/4.2.txt
@@ -51,6 +51,9 @@ Minor features
 * The ``admin/base.html`` template now has a new block ``nav-breadcrumbs``
   which contains the navigation landmark and the ``breadcrumbs`` block.

+* :attr:`.ModelAdmin.list_editable` now uses atomic transactions when making
+  edits.
+
 :mod:`django.contrib.admindocs`
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

...

I must have skimmed over it, woops! Well, at least I had verified that the behaviour change was expected. I opted to update the test's expected query count and comment:

class BookAdminTest(TestCase):
    ...

    def test_save_change_list_view_num_of_queries(self) -> None:
        ...  # Some setup

        with self.assertNumQueries(9):
            """
            1. SAVEPOINT ...
            2. SELECT "django_session" ...
            3. SELECT "core_user" ...
            4. SELECT COUNT(*) AS "__count" FROM "library_book"
            5. SELECT COUNT(*) AS "__count" FROM "library_book"
            6. SELECT "library_book" ...
            7. SAVEPOINT ...
            8. RELEASE SAVEPOINT ...
            9. RELEASE SAVEPOINT ...
            """
            self.client.post("/admin/library/book/", ...)

Search by behaviour with git bisect

Searching through the log and diff is hard when you don't know which files to look at or strings to search for. If you're observing a behaviour change and don't know its cause, try using git bisect to find the responsible commit.

See my git bisect basics post for an introduction to the command. Here, we'll discuss the specifics of bisecting Django in your project.

First, use an editable install of your local Django repository within the target project. With Pip, use pip install -e:

$ python -m pip install -e ~/Projects/django

Replace ~/Projects with the appropriate path to the Django repository.

Second, ensure you can run your behaviour test on both Django versions. That behaviour test might be loading a page under ./manage.py runserver, a unit test with ./manage.py test or pytest, or some other command. Your project must work sufficiently on both Django versions before you can bisect between them.

Switch your Django repository to the older version:

$ git switch stable/4.2.x

Then, run the behaviour test in your project:

$ ./manage.py runserver
...
[24/Apr/2024 05:25:59] "GET / HTTP/1.1" 200 1797

Similarly, repeat on the newer version:

$ git switch stable/5.0.x
$ ./manage.py runserver
...
[24/Apr/2024 05:26:25] "GET / HTTP/1.1" 200 1761

If your test does not run smoothly on both versions, modify the project until it does. Typically, this means acting on deprecation warnings from the old version. But in the worst case, you may need to fork code between the old and new versions, especially if you're trying to upgrade many versions. Try this pattern for forking based on Django's version tuple:

import django

if django.VERSION >= (5, 0):
    # do the new thing
    ...
else:
    # do the old thing
    ...

Third, run the bisect. In the Django repository, start the bisect and label the old and new versions like so:

$ git bisect start
$ git bisect old $(git merge-base main stable/4.2.x)
$ git bisect new stable/5.0.x

Do the usual thing of iterating with the old and new subcommands until you find the responsible commit. Remember to finish up with git bisect reset.

Finally, roll back your project to use the non-editable install of Django. For example, with Pip:

$ python -m pip install -r requirements.txt

Fin

See you in the Django' Git log,

-Adam

24 Apr 2024 4:00am GMT

23 Apr 2024

feedDjango community aggregator: Community blog posts

Django News - [Resend of #228] PyPI Expanding Trusted Publisher Support - Apr 22nd 2024

Introduction

Hi everyone,

We apologize to anyone who didn't receive the Django News Newsletter Issue #228 last Friday and to anyone who just received a duplicate edition. Issue #229 is a re-send of what everything should have received last Friday.

Last week, our newsletter provider had a hiccup, and we estimate that less than 10% of our subscribers received their weekly Friday edition of Django News. We felt terrible that you didn't receive it and that two conference CFPs will have ended before our next newsletter goes out.

We decided our best bet was to re-send everyone an updated Monday edition of Django News and apologize again for Friday's mishap.

Please note that Wagtail Space US's CFP ends today, April 22nd (hours left), and DjangoCon US's CFP ends Wednesday, April 24th (less than two days left).

Jeff and Will

Django Newsletter

News

Don't Miss Out: Last Call for DjangoCon US 2024 Talk Proposals!

Have you submitted your talk or tutorial for DjangoCon US 2024, in beautiful Durham, North Carolina, USA?

This is your last call to submit a talk or tutorial. The CFP deadline is April 24, 2024, at 12 PM EDT.

djangocon.us

PyPI: Expanding Trusted Publisher Support

PyPI added GitLab CI/CD, Google Cloud, and ActiveState as Trusted Publishing providers.

pypi.org

Django Software Foundation

DSF Board meeting minutes for April 11, 2024

Here are the DSF Board's meeting minutes for April 11, 2024.

djangoproject.com

Updates to Django

Today 'Updates to Django' is presented by Raffaella Suardini from Djangonaut Space!

Last week we had 10 pull requests merged into Django by 5 different contributors - including 1 first-time contributor! Congratulations to Aleksander Milinkevich for having their first commits merged into Django - welcome on board!

Coming in Django 5.0.5 (expected May 6th):

Django Newsletter

Wagtail CMS

Wagtail Space CFP deadline is April 22

It's the last call to speak this summer at Wagtail Space US 2024.

Wagtail Space US 2024 - Call For Proposals

Wagtail Space Virtual Talks 2024 - Call For Proposals

wagtail.space

Sponsored Ad

Free Trial of Scout APM Today!

Need answers to your Django app questions fast? Avoid the hassle of talking with a sales rep and the long wait times of large support teams, and choose Scout APM. Get Django insights in less than 4 minutes with Scout APM.

scoutapm.com

Articles

7 simple examples using Django GeneratedField

Django 5.0 added a new feature, GeneratedField, which allows us to auto-calculate database fields. This article shows seven short examples of how to use it, so the database performs calculations exceptionally quickly.

photondesigner.com

Django from first principles, part 3

In the third installment of his series on constructing a comprehensive Django project from a single file, Eric Matthes explores enhancing your homepage with templates.

mostlypython.com

Building forms with the Django admin

A look at multiple ways, including over time, to build and style forms in the Django admin.

406.ch

Styling a Django RSS Feed

A straightforward way to style your RSS feed in a Django app.

hyteck.de

Enforcing conventions in Django projects with introspection

Some code and tips to combine Python and Django introspection APIs to enforce naming conventions in your Django models.

lukeplant.me.uk

Tutorials

Building a Voice Notes App with Django and OpenAI

In this tutorial, you will learn how to build a voice notes app using Django and OpenAI for speech-to-text conversion. Additionally, AlpineJS will manage the state on the front end.

circumeo.io

Videos

Django 2024: The Latest Development Trends

Scheduled for Apr 25, 2024. Tune in to our upcoming livestream, where we'll take you through the latest Django Developers Survey results based on responses from 4,000 Django developers.

youtube.com

Understanding Wasm: How We Got Here by Chris Dickinson @ Wasm I/O 2024

Let's put Wasm and the problems it solves into historical context: what are we trying to solve, and for whom? What has been tried before? What makes this effort more likely to succeed? We'll examine the history of virtual machines, operating systems, & hypervisors from the 1960s through the 2010s.

youtube.com

Podcasts

Django Chat #161: Kraken - Çağıl Uluşahin Sönmez

Çağıl is a Lead Backend Engineer at Kraken Tech, Django Software Foundation Vice President, and Django London Meetup co-organizer. We discuss her background studying computer science in Turkey, organizing DjangoGirls and Python events in Istanbul, and her current work today.

djangochat.com

Python Test #218: Balancing test coverage with test costs

Nicole is a software engineer and writer, and recently wrote about the trade-offs we make when deciding which tests to write and how much testing is enough.

pythontest.com

Django News Jobs

New jobs for this week!

Senior AI Engineer (f/m/d) at 1&1 🆕

Michigan Online Software Engineer at University of Michigan

Web developer at der Freitag Mediengesellschaft

Backend Software Architect, EarthRanger (Contract Opportunity) at AI2

Senior Software Engineer (backend) - IASO at Bluesquare

Remote Full-stack Python Developer at Scopic

Django Developer at The Developer Society

Django Newsletter

Projects

matthiask/django-translated-fields

Django model translation without magic-inflicted pain.

github.com

inkandswitch/tiny-essay-editor

Simple markdown editor w inline comments, on latest auto merge stack.

github.com

Sponsorship

🌷 Spring Newsletter Sponsorships

Want to reach over 3,765+ active Django developers?

Full information is available on the sponsorship page.

django-news.com


This RSS feed is published on https://django-news.com/. You can also subscribe via email.

23 Apr 2024 5:58am GMT

19 Apr 2024

feedDjango community aggregator: Community blog posts

Django News - PyPI Expanding Trusted Publisher Support - Apr 19th 2024

News

Don't Miss Out: Last Call for DjangoCon US 2024 Talk Proposals!

Have you submitted your talk or tutorial for DjangoCon US 2024, in beautiful Durham, North Carolina, USA?

This is your last call to submit a talk or tutorial. The CFP deadline is April 24, 2024, at 12 PM EDT.

djangocon.us

PyPI: Expanding Trusted Publisher Support

PyPI added GitLab CI/CD, Google Cloud, and ActiveState as Trusted Publishing providers.

pypi.org

Django Software Foundation

DSF Board meeting minutes for April 11, 2024

Here are the DSF Board's meeting minutes for April 11, 2024.

djangoproject.com

Updates to Django

Today 'Updates to Django' is presented by Raffaella Suardini from Djangonaut Space!

Last week we had 10 pull requests merged into Django by 5 different contributors - including 1 first-time contributor! Congratulations to Aleksander Milinkevich for having their first commits merged into Django - welcome on board!

Coming in Django 5.0.5 (expected May 6th):

Django Newsletter

Wagtail CMS

Wagtail Space CFP deadline is April 22

It's the last call to speak this summer at Wagtail Space US 2024.

Wagtail Space US 2024 - Call For Proposals

Wagtail Space Virtual Talks 2024 - Call For Proposals

wagtail.space

Sponsored Ad

Free Trial of Scout APM Today!

Need answers to your Django app questions fast? Avoid the hassle of talking with a sales rep and the long wait times of large support teams, and choose Scout APM. Get Django insights in less than 4 minutes with Scout APM.

scoutapm.com

Articles

7 simple examples using Django GeneratedField

Django 5.0 added a new feature, GeneratedField, which allows us to auto-calculate database fields. This article shows seven short examples of how to use it so the database performs calculations extremely quickly.

photondesigner.com

Django from first principles, part 3

In the third installment of his series on constructing a comprehensive Django project from a single file, Eric Matthes explores enhancing your homepage with templates.

mostlypython.com

Building forms with the Django admin

A look at multiple ways, including over time, to build and style forms in the Django admin.

406.ch

Styling a Django RSS Feed

A straightforward way to style your RSS feed in a Django app.

hyteck.de

Enforcing conventions in Django projects with introspection

Some code and tips to combine Python and Django introspection APIs to enforce naming conventions in your Django models.

lukeplant.me.uk

Tutorials

Building a Voice Notes App with Django and OpenAI

In this tutorial, you will learn how to build a voice notes app using Django and OpenAI for speech-to-text conversion. Additionally, AlpineJS will manage the state on the front end.

circumeo.io

Videos

Django 2024: The Latest Development Trends

Scheduled for Apr 25, 2024. Tune in to our upcoming livestream, where we'll take you through the latest Django Developers Survey results based on responses from 4,000 Django developers.

youtube.com

Understanding Wasm: How We Got Here by Chris Dickinson @ Wasm I/O 2024

Let's put Wasm and the problems it solves into historical context: what are we trying to solve, and for whom? What has been tried before? What makes this effort more likely to succeed? We'll examine the history of virtual machines, operating systems, & hypervisors from the 1960s through the 2010s.

youtube.com

Podcasts

Django Chat #161: Kraken - Çağıl Uluşahin Sönmez

Çağıl is a Lead Backend Engineer at Kraken Tech, Django Software Foundation Vice President, and Django London Meetup co-organizer. We discuss her background studying computer science in Turkey, organizing DjangoGirls and Python events in Istanbul, and her current work today.

djangochat.com

Python Test #218: Balancing test coverage with test costs

Nicole is a software engineer and writer, and recently wrote about the trade-offs we make when deciding which tests to write and how much testing is enough.

pythontest.com

Django News Jobs

New jobs for this week!

Senior AI Engineer (f/m/d) at 1&1 🆕

Michigan Online Software Engineer at University of Michigan

Web developer at der Freitag Mediengesellschaft

Backend Software Architect, EarthRanger (Contract Opportunity) at AI2

Senior Software Engineer (backend) - IASO at Bluesquare

Remote Full-stack Python Developer at Scopic

Django Developer at The Developer Society

Django Newsletter

Projects

matthiask/django-translated-fields

Django model translation without magic-inflicted pain.

github.com

inkandswitch/tiny-essay-editor

Simple markdown editor w inline comments, on latest auto merge stack.

github.com

Sponsorship

🌷 Spring Newsletter Sponsorships

Want to reach over 3,750+ active Django developers? Full information is available on the sponsorship page.

django-news.com


This RSS feed is published on https://django-news.com/. You can also subscribe via email.

19 Apr 2024 3:00pm GMT

18 Apr 2024

feedDjango community aggregator: Community blog posts

Why Django and why not Flask?

Why would someone pick Django over Flask? That's the question that I got on stream and here is my answer.

18 Apr 2024 5:00am GMT