fidzu : django

I use this a lot. It has served me very well. The code:

import hashlib
import functools

import markus  # optional
from django.core.cache import cache
from django import http
from django.utils.encoding import force_bytes, iri_to_uri

metrics = markus.get_metrics(__name__)  # optional


def json_response_cache_page_decorator(seconds):
    """Cache only when there's a healthy http.JsonResponse response."""

    def decorator(func):

        @functools.wraps(func)
        def inner(request, *args, **kwargs):
            cache_key = 'json_response_cache:{}:{}'.format(
                func.__name__,
                hashlib.md5(force_bytes(iri_to_uri(
                    request.build_absolute_uri()
                ))).hexdigest()
            )
            content = cache.get(cache_key)
            if content is not None:

                # metrics is optional
                metrics.incr(
                    'json_response_cache_hit',
                    tags=['view:{}'.format(func.__name__)]
                )

                return http.HttpResponse(
                    content,
                    content_type='application/json'
                )
            response = func(request, *args, **kwargs)
            if (
                isinstance(response, http.JsonResponse) and
                response.status_code in (200, 304)
            ):
                cache.set(cache_key, response.content, seconds)
            return response

        return inner

    return decorator

To use it simply add to Django view functions that might return a http.JsonResponse. For example, something like this:

@json_response_cache_page_decorator(60)
def search(request):
    q = request.GET.get('q')
    if not q:
        return http.HttpResponseBadRequest('no q')
    results = search_database(q)
    return http.JsonResponse({
        'results': results,
    })

The reasons I use this instead of django.views.decorators.cache.cache_page() is because of a couple of reasons.

• cache_page generates cache keys that don't contain the view function name.
• cache_page tries to cache the whole http.HttpResponse instance which can't be serialized if you use the msgpack serializer.
• cache_page also sends Cache-Control headers which is not always what you want.
• Not possible to inject your own custom code such as my usage of metrics.

Disclaimer: This snippet of code comes from a side-project that has a very specific set of requirements. They're rather unique to that project and I have a full picture of the needs. E.g. I know what specific headers matter and don't matter. Your project might be different. For example, perhaps you don't have markus to handle your metrics. Or perhaps you need to re-write the query string for something to normalize the cache key differently. Point being, take the snippet of code as inspiration when you too find that django.views.decorators.cache.cache_page() isn't good enough for your Django view functions.

20 Jun 2018 6:55pm GMT

Back in March I announced the release of a couple security-related projects for Django, one that implements the Referrer-Policy header, and one that uses the Pwned Passwords database of Have I Been Pwned to check users' passwords.

Today I've bumped the version and rolled a new release of pwned-passwords-django; if you're reading this, version 1.2 is on the Python Package Index, and is only a pip install away. And, of course, there's full documentation available ...

Read full entry

19 Jun 2018 4:52am GMT

Website experiences need to be consistent as much as they need to be well thought out and aesthetically pleasing. Structure, visual design, user interactions, and accessibility concerns are among many considerations that go into building quality websites. While achieving consistency of experience and implementation is an essential goal of web development, efficiency of execution is also very important. An efficient workflow means this consistent experience doesn't require redoing work across the site.

This post is about efficient consistency when building forms across your site.

Django helps you build forms, but one size doesn't fit all. It can render your forms on its own, or you can take more control of the form markup in your HTML. When Django renders your forms, you adhere to its defaults and assumptions. When those don't match your site's designs or other requirements, you have to do it yourself. But you can squeeze more flexibility out of Django's own form rendering. This lets you match your form styles and implementations site-wide without losing the valuable tools Django has out-of-the-box to make form rendering easier.

Why change what Django does?

Maybe you've always been fine using the forms exactly as Django renders them for you. Or, maybe you've been building custom forms in Django for so long you don't see what's wrong with providing your own widget classes or adding the extra attributes to your fields. And, of course, you can get a lot of customization out of simply re-styling the form pieces in CSS after Django has done its rendering, so you have lots of options for flexibility.

There have been a lot of situations where I need to change how lots of forms are rendered, usually across an entire site:

• Accessibility requirements stipulate aria-required and other attributes
• Design or CSS frameworks necessitate changes to an input's markup
• Design or CSS frameworks necessitate changes to all inputs, like common attributes or even common event triggers
• I need to replace the traditional file input with a smarter widget
• I also need to replace built-in date and time inputs

None of the above are difficult to account for. The problem we're looking at is applying this list of concerns, and more, to all form fields on an entire site, and that often includes forms that come from third-party Django apps where you don't even have access to change the forms themselves. (Short of forking all your third-party apps, which is a really crappy proposition.)

These situations are also increasingly difficult to deal with on existing sites, because the larger the site gets, the more forms it has.

Ideally, make the changes once

Django and Python have some assumptions and guidelines about code. One of the most important ones is removing verbosity and redundancy. The current approaches do neither, so let's find a better way.

We'll look at two things we can do. The first is the larger impact on our flexibility. The second is a smaller, but also useful method of customizing form defaults.

Django widget templates

As part of the 1.11 release of Django, widgets are now rendered by templates, just like everything else. This gives you the opportunity to create your own widgets much more easily. But, it also gives us an opportunity to override the templates Django uses for the built-in widgets it comes with.

Obviously, this advice assumes that your project has been upgraded to the 1.11 release of Django or higher.

There is a new type of component in a Django project, the Form Renderer. You can imagine this is very much related to what we're trying to do! There is a setting to select which Form Renderer you use, and Django itself comes with three choices, but you could implement your own. For our purposes, one of the built-in renderers will work, just not the default.

The default form renderer is the DjangoTemplates renderer, which sounds like it would do exactly what we need, but does not. This renderer uses its own template engine, separate from your project settings. It will load the default widgets first and then look in all your applications for widget templates for all your custom widgets.

We'll use the TemplateSettings renderer, instead, which loads templates exactly as the rest of our project is configured.

  
FORM_RENDERER = 'django.forms.renderers.TemplatesSetting'

Now that the form rendering can be configured with regard to templates, let's look at some settings that will load our widget templates in the order we want. Some of this can be changed to adapt to your needs, but this is what worked for our project:

  
"DIRS": [
    "project/templates/",
],

"loaders": [
    "django.template.loaders.filesystem.Loader",
    "django.template.loaders.app_directories.Loader",
],

We're telling Django to first look for templates in our project's own templates directory, which is where we're going to put our widget templates. You could also override widgets in an app, but for overriding the defaults I think it is appropriate to do so in a global context.

One of the important overrides we made was to change how attributes inside the input tags are rendered. All the default widget templates exist in django/forms/widgets/ under any templates directory they're being looked up in, so our project has the template project/templates/django/forms/widgets/attrs.html.

Of course, we aren't going to override all the default widget templates. Although, you could, if you wanted to! For the templates we don't override, we still want the renderer to look in Django's form app. To do this, we need to add django.forms to our INSTALLED_APPS list, but we put it at the end, so that any overrides that might exist inside other apps can be found first and the defaults are always the last ones used.

  
INSTALLED_APPS = [
    ...

    'django.forms', # must be last!
]

What did we do by overriding these widget templates?

• We added a small onchange handler to toggle a class on any input when it has a value, so our CSS can target empty or non-empty inputs. Very useful!
• We added accessibility tags to all our inputs without exception.
• We changed how our radio and checkbox lists were rendered to remove the colon in the labels because that didn't match our design.

The new Form Rendering system in Django 1.11 adds a lot of control we didn't have before, and it was really fun to explore it and see how it could help us. Overall, I'm extremely happy with the result.

A trick for a little more customization

Django widget templates are a supported feature, and there are lots of other features in the framework that make tailoring your setup to a project's needs really easy. That said, what do you do about things you can't customize, but have a good case for?

As an example, let's look at one more thing Django forms do out-of-the-box. When Django renders a form for you, it renders a series of both labels and fields. We've talked about customizing the fields, but the labels are actually external to the widgets and their templates.

I'm going to use a silly example, but a real one. In our design, we did not like the colons Django includes as a suffix of every label. Of course, there were lots of ways around this. I could create my forms with an empty label_suffix option, for example. But, if you've read this far, you'll know that doing anything more than once is too often for me.

There is no setting you can use to change the default label suffix globally across a project. But there is a trick you can employ to make the base Form class that all your forms derive from use a different default: just replace the base Form class with one that does what you want!

  
from django import forms

class BaseForm(forms.Form):
    def __init__(self, *args, **kwargs):
        kwargs.setdefault('label_suffix', '')  
        super(BaseForm, self).__init__(*args, **kwargs)

forms.Form = BaseForm

This is a roundabout way to accomplish our little goal, and I admit it is a trivial goal. You might find other reasons to do this, with other changes that you want to make across all the forms on your site. A fair warning is due, however. This is monkeypatching, it is generally frowned upon, and you have to be careful with changes that will affect code you didn't write.

I think this is a light and safe use, but be careful if you do anything more with it!

Conclusion

We haven't gone over anything complex here. Using some simple tricks and an easy application of new features supported by Django can go a long way towards creating great form experiences across your site.

Hopefully, this helps you work faster with even better results for you, your clients, and your users.

18 Jun 2018 6:00pm GMT