15 Jan 2019
Django community aggregator: Community blog posts
The Future of Typography: An Introduction to Variable Fonts
Designers, imagine a world in which the number of fonts used on a website or app isn't limited by load speed, or by the availability of styles in a font family. You'd no longer be limited to a few weights (light, regular, bold) and widths (condensed, regular, expanded). Instead, you could exercise a fine degree of control over all of these properties. The post The Future of Typography: An Introduction to Variable Fonts appeared first on Distillery.
15 Jan 2019 11:43pm GMT
13 Jan 2019
Django community aggregator: Community blog posts
How to Dockerize Django and Postgres
Docker is an invaluable tool that makes things like setting up your local development environment, continuous integration, and deployment a breeze. Docker is good to set up early on for any application, but it is never too late. With the help of Docker Compose, we are going to set up a Django application to install … Continue reading How to Dockerize Django and Postgres
The post How to Dockerize Django and Postgres appeared first on concise coder.
13 Jan 2019 4:27am GMT
11 Jan 2019
Django community aggregator: Community blog posts
How to Create PDF Documents with Django in 2019
If you've read my Web Development with Django Cookbook, you might remember a recipe for creating PDF documents using Pisa xhtml2pdf. Well, this library does its job, but it supports only a subset of HTML and CSS features. For example, for multi-column layouts, you have to use tables, like it's 1994.
I needed some fresh and flexible option to generate donation receipts for the donation platform www.make-impact.org and reports for the strategic planner 1st things 1st I have been building. After a quick research I found another much more suitable library. It's called WeasyPrint. In this article, I will tell you how to use it with Django and what's valuable in it.
Features
WeasyPrint uses HTML and CSS 2.1 to create pixel-perfect, or let's rather say point-perfect, PDF documents. WeasyPrint doesn't use WebKit or Gecko but has its own rendering engine. As a proof that it works correctly, it passes the famous among web developers Acid2 test which was created back in the days before HTML5 to check how compatible browsers are with CSS 2 standards.
All supported features and unsupported exceptions are listed in the documentation. But my absolute favorites are these:
- Layouts with floated elements. You don't have to use tables anymore if you want to have the recipient address on the left side and the sender information on the right side in a letter, or if you want to have the main content and the side notes in an exercise book. Just use floated elements.
- Working links. The generated document can have clickable links to external URLs and internal anchors. You can straightforwardly create a clickable table of contents or a banner that leads back to your website.
- Support for web fonts. With the wide variety of embeddable web fonts, your documents don't need to look boring anymore. Why not write titles in elegant cursive or in bold western letters?
- Background images. By default, when you print an HTML page, all foreground images get printed, but the backgrounds are skipped. When you generate a PDF document for printing, you can show background images anywhere, even in the margins of the printed page.
- SVG kept as vector images. When you have diagrams and graphics in a PDF document, you usually want to preserve the quality of the lines. Even if they look good on the screen, raster images might be not what you want, because on a printed page the resolution will differ and the quality can be lost. WeasyPrint keeps SVG images as vector images, so you have the highest possible quality in the prints.
Important Notes
WeasyPrint needs Python 3.4 or newer. That's great for new Django projects, but might be an obstacle if you want to integrate it into an existing website running on Python 2.7. Can it be the main argumentation for you to upgrade your old Django projects to the new Python version?
WeasyPrint is dependent on several OS libraries: Pango, GdkPixbuf, Cairo, and Libffi. In the documentation, there are understandable one-line instructions how to install them on different operating systems. You can have a problem only if you don't have full control of the server where you are going to deploy your project.
If you need some basic headers and footers for all pages, you can use @page CSS selector for that. If you need extended headers and footers for each page, it's best to combine the PDF document out of separate HTML documents for each page. Examples follow below.
The fun fact, Emojis are drawn using some weird raster single-color font. I don't recommend using them in your PDFs unless you replace them with SVG images.
Show Me the Code
A technical article is always more valuable when it has some quick code snippets to copy and paste. Here you go!
Simple PDF View
This snippet generates a donation receipt and shows it directly in the browser. Should the PDF be downloadable immediately, change content disposition from inline to attachment.
# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
from django.http import HttpResponse
from django.template.loader import render_to_string
from django.utils.text import slugify
from django.contrib.auth.decorators import login_required
from weasyprint import HTML
from weasyprint.fonts import FontConfiguration
from .models import Donation
@login_required
def donation_receipt(request, donation_id):
donation = get_object_or_404(Donation, pk=donation_id, user=request.user)
response = HttpResponse(content_type="application/pdf")
response['Content-Disposition'] = "inline; filename={date}-{name}-donation-receipt.pdf".format(
date=donation.created.strftime('%Y-%m-%d'),
name=slugify(donation.donor_name),
)
html = render_to_string("donations/receipt_pdf.html", {
'donation': donation,
})
font_config = FontConfiguration()
HTML(string=html).write_pdf(response, font_config=font_config)
return response
Page Configuration Using CSS
Your PDF document can have a footer with an image and text on every page, using background-image and content properties:
{% load staticfiles i18n %}
<link href="https://fonts.googleapis.com/css?family=Playfair+Display:400,400i,700,700i,900" rel="stylesheet" />
<style>
@page {
size: "A4";
margin: 2.5cm 1.5cm 3.5cm 1.5cm;
@bottom-center {
background: url({% static 'site/img/logo-pdf.svg' %}) no-repeat center top;
background-size: auto 1.5cm;
padding-top: 1.8cm;
content: "{% trans "Donation made via www.make-impact.org" %}";
font: 10pt "Playfair Display";
text-align: center;
vertical-align: top;
}
}
</style>
Pagination
You can show page numbers in the footer using CSS as follows.
@page {
margin: 3cm 2cm;
@top-center {
content: "Documentation";
}
@bottom-right {
content: "Page " counter(page) " of " counter(pages);
}
}
Horizontal Page Layout
You can rotate the page to horizontal layout with size: landscape.
@page {
size: landscape;
}
HTML-based Footer
Another option to show an image and text in the header or footer on every page is to use an HTML element with position: fixed. This way you have more flexibility about formatting, but the element on all your pages will have the same content.
<style>
footer {
position: fixed;
bottom: -2.5cm;
width: 100%;
text-align: center;
font-size: 10pt;
}
footer img {
height: 1.5cm;
}
</style>
<footer>
{% with website_url="https://www.make-impact.org" %}
<a href="{{ website_url }}">
<img alt="" src="{% static 'site/img/logo-contoured.svg' %}" />
</a><br />
{% blocktrans %}Donation made via <a href="{{ website_url }}">www.make-impact.org</a>{% endblocktrans %}
{% endwith %}
</footer>
Document Rendering from Page to Page
When you need to have a document with complex unique headers and footers, it is best to render each page as a separate HTML document and then to combine them into one. This is how to do that:
def letter_pdf(request, letter_id):
letter = get_object_or_404(Letter, pk=letter_id)
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = (
'inline; '
f'filename={letter.created:%Y-%m-%d}-letter.pdf'
)
COMPONENTS = [
'letters/pdf/cover.html',
'letters/pdf/page01.html',
'letters/pdf/page02.html',
'letters/pdf/page03.html',
]
documents = []
font_config = FontConfiguration()
for template_name in COMPONENTS:
html = render_to_string(template_name, {
'letter': letter,
})
document = HTML(string=html).render(font_config=font_config)
documents.append(document)
all_pages = [page for document in documents for page in document.pages]
documents[0].copy(all_pages).write_pdf(response)
return response
Final Thoughts
I believe that WeasyPrint could be used not only for invoices, tickets, or booking confirmations but also for online magazines and small booklets. If you want to see PDF rendering with WeasyPrint in action, make a donation to your chosen organization at www.make-impact.org (when it's ready) and download the donation receipt. Or check the demo account at my.1st-things-1st.com and find the button to download the results of a prioritization project as PDF document.
Cover photo by Daniel Korpai.
11 Jan 2019 9:10pm GMT
Django Tips #7: Function-Based Views vs Class-Based Views
An overview of the history and pros/cons of each approach.
11 Jan 2019 2:28pm GMT
09 Jan 2019
Django community aggregator: Community blog posts
How to Use Django Bulk Inserts for Greater Efficiency
It's been awhile since we last discussed bulk inserts on the Caktus blog. The idea is simple: if you have an application that needs to insert a lot of data into a Django model - for example a background task that processes a CSV file (or some other text file) - it pays to "chunk" those updates to the database so that multiple records are created through a single database operation. This reduces the total number of round-trips to the database, something my colleague Dan Poirier discussed in more detail in the post linked above.
Today, we use Django's Model.objects.bulk_create() regularly to help speed up operations that insert a lot of data into a database. One of those projects involves processing a spreadsheet with multiple tabs, each of which might contain thousands or even tens of thousands of records, some of which might correspond to multiple model classes. We also need to validate the data in the spreadsheet and return errors to the user as quickly as possible, so structuring the process efficiently helps to improve the overall user experience.
While it's great to have support for bulk inserts directly in Django's ORM, the ORM does not provide much assistance in terms of managing the bulk insertion process itself. One common pattern we found ourselves using for bulk insertions was to:
- build up a list of objects
- when the list got to a certain size, call bulk_create()
- make sure any objects remaining (i.e., which might be fewer than the chunk size of prior calls to bulk_create()) are inserted as well
Since for this particular project we needed to repeat the same logic for a number of different models in a number of different places, it made sense to abstract that into a single class to handle all of our bulk insertions. The API we were looking for was relatively straightforward:
- Set bulk_mgr = BulkCreateManager(chunk_size=100) to create an instance of our bulk insertion helper with a specific chunk size (the number of objects that should be inserted in a single query)
- Call bulk_mgr.add(unsaved_model_object) for each model instance we needed to insert. The underlying logic should determine if/when a "chunk" of objects should be created and does so, without the need for the surrounding code to know what's happening. Additionally, it should handle objects from any model class transparently, without the need for the calling code to maintain separate object lists for each model.
- Call bulk_mgr.done() after adding all the model objects, to insert any objects that may have been queued for insertion but not yet inserted.
Without further ado, here's a copy of the helper class we came up with for this particular project:
from collections import defaultdict from django.apps import apps class BulkCreateManager(object): """ This helper class keeps track of ORM objects to be created for multiple model classes, and automatically creates those objects with `bulk_create` when the number of objects accumulated for a given model class exceeds `chunk_size`. Upon completion of the loop that's `add()`ing objects, the developer must call `done()` to ensure the final set of objects is created for all models. """ def __init__(self, chunk_size=100): self._create_queues = defaultdict(list) self.chunk_size = chunk_size def _commit(self, model_class): model_key = model_class._meta.label model_class.objects.bulk_create(self._create_queues[model_key]) self._create_queues[model_key] = [] def add(self, obj): """ Add an object to the queue to be created, and call bulk_create if we have enough objs. """ model_class = type(obj) model_key = model_class._meta.label self._create_queues[model_key].append(obj) if len(self._create_queues[model_key]) >= self.chunk_size: self._commit(model_class) def done(self): """ Always call this upon completion to make sure the final partial chunk is saved. """ for model_name, objs in self._create_queues.items(): if len(objs) > 0: self._commit(apps.get_model(model_name))
You can then use this class like so:
import csv with open('/path/to/file', 'rb') as csv_file: bulk_mgr = BulkCreateManager(chunk_size=20) for row in csv.reader(csv_file): bulk_mgr.add(MyModel(attr1=row['attr1'], attr2=row['attr2'])) bulk_mgr.done()
I tried to simplify the code here as much as possible for the purposes of this example, but you can obviously expand this as needed to handle multiple model classes and more complex business logic. You could also potentially put bulk_mgr.done() in its own finally: or except ExceptionType: block, however, you should be careful not to write to the database again if the original exception is database-related.
Another useful pattern might be to design this as a context manager in Python. We haven't tried that yet, but you might want to.
Good luck with speeding up your Django model inserts, and feel free to post below with any questions or comments!
09 Jan 2019 7:00pm GMT
Customizing Django REST API Serializers
Adding extra fields to Serializers
# serializers.py
from rest_framework import serializers
from test_app.models import Product
class ProductSerializer(serializers.Serializer):
product_id = serializers.IntegerField()
product = serializers.SerializerMethodField('get_product_name')
def get_product_name(self, obj):
if obj.get("product_id"):
obj_product = Product.objects.filter(id=obj.get("product_id")).first()
if obj_product:
return obj_product.name
return None
# API views.py
from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework import status
from test_app.serializers import ProductsSerializer
@api_view(['GET'])
def list_products(request):
response_data = {}
response_data["data"] = ProductSerializer(
[{"program": i} for i in [1, 2, 3]],
many=True
).data
return Response(response_data, status=status.HTTP_200_OK)
Passing extra arguments to Serializer Class in Django Rest Framework
Suppose we have searlizer called 'CollegeStudentsSerializer' which should return all students, branches details of a logged-in user college
# serializers.py
from rest_framework import serializers
from test_app.models import Student, Branch
class StudentSerializer(serializers.ModelSerializer):
class Meta:
model = Student
fields = ['id', 'first_name', 'last_name', 'branch']
class BranchSerializer(serializers.ModelSerializer):
class Meta:
model = Branch
fields = ['id', 'name', 'code']
class CollegeDetailsSerializer(serializers.Serializer):
students = serializers.SerializerMethodField('get_students')
branches = serializers.SerializerMethodField('get_branches')
def __init__(self, *args, **kwargs):
context = kwargs.pop("context")
self.college_id = context.get('college_id')
super(CollegeDetailsSerializer, self).__init__(*args, **kwargs)
def get_students(self, obj):
return StudentSerializer(
Student.objects.filter(college_id=self.college_id),
many=True
).data
def get_branches(self, obj):
return BranchSerializer(
Branch.objects.filter(college_id=self.college_id),
many=True
).data
# API views.py
from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework import status
from test_app.serializers import CollegeDetailsSerializer
@api_view(['GET'])
def students_list(request):
response_data = {"data": []}
if request.user and request.user.college_id:
response_data["data"] = CollegeDetailsSerializer(
context={"college_id": request.user.college_id},
many=True
).data
return Response(response_data, status=status.HTTP_200_OK)
09 Jan 2019 6:00am GMT
08 Jan 2019
Django community aggregator: Community blog posts
7 Awesome Augmented Reality Solutions
Augmented reality, also known as AR, is increasingly everywhere. Businesses across industries are turning to augmented reality app development to help them create new ways to engage their customers. Educators are using AR to develop new ways of teaching, and medical professionals are using it to transform the way they approach treatment. People of all ages and backgrounds are trying out the built-in AR functionalities of their smartphones. The post 7 Awesome Augmented Reality Solutions appeared first on Distillery.
08 Jan 2019 11:47pm GMT
05 Jan 2019
Django community aggregator: Community blog posts
Marginal Gain #1: Make Common Tasks Easy
Marginal gains are minor improvements that are relatively easy to make. A single marginal gain does not have a meaningful effect, but several can aggregate into significant progress. I first read about this concept in James Clear's Atomic Habits, and was inspired to apply this philosophy to software development. For a better understanding of what … Continue reading Marginal Gain #1: Make Common Tasks Easy
The post Marginal Gain #1: Make Common Tasks Easy appeared first on concise coder.
05 Jan 2019 6:00pm GMT
04 Jan 2019
Django community aggregator: Community blog posts
Distillery Makes the Entrepreneur 360TM for the Second Year in a Row!
Achieving success in business is no easy task. Sustaining that success is even harder. That's why we're excited to share another milestone marking sustained success for Distillery. For the second year in a row, we've been honored on the Entrepreneur 360! The post Distillery Makes the Entrepreneur 360TM for the Second Year in a Row! appeared first on Distillery.
04 Jan 2019 12:07am GMT
03 Jan 2019
Django community aggregator: Community blog posts
How to Use Date Picker with Django
In this tutorial we are going to explore three date/datetime pickers options that you can easily use in a Django project. We are going to explore how to do it manually first, then how to set up a custom widget and finally how to use a third-party Django app with support to datetime pickers.
- Introduction
- Tempus Dominus Bootstrap 4
- XDSoft DateTimePicker
- Fengyuan Chen's Datepicker
- Conclusions
Introduction
The implementation of a date picker is mostly done on the front-end.
The key part of the implementation is to assure Django will receive the date input value in the correct format, and also that Django will be able to reproduce the format when rendering a form with initial data.
We can also use custom widgets to provide a deeper integration between the front-end and back-end and also to promote better reuse throughout a project.
In the next sections we are going to explore following date pickers:
Tempus Dominus Bootstrap 4 Docs Source
XDSoft DateTimePicker Docs Source
Fengyuan Chen's Datepicker Docs Source
Tempus Dominus Bootstrap 4
This is a great JavaScript library and it integrate well with Bootstrap 4. The downside is that it requires moment.js and sort of need Font-Awesome for the icons.
It only make sense to use this library with you are already using Bootstrap 4 + jQuery, otherwise the list of CSS and JS may look a little bit overwhelming.
To install it you can use their CDN or download the latest release from their GitHub Releases page.
If you downloaded the code from the releases page, grab the processed code from the build/ folder.
Below, a static HTML example of the datepicker:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Static Example</title>
<!-- Bootstrap 4 -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/css/bootstrap.min.css" integrity="sha384-GJzZqFGwb1QTTN6wy59ffF1BuGJpLSa9DkKMp0DgiMDm4iYMj70gZWKYbI706tWS" crossorigin="anonymous">
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.6/umd/popper.min.js" integrity="sha384-wHAiFfRlMFy6i5SRaxvfOCifBUQy1xHdJ/yoi7FRNXMRBu5WHdZYu1hA6ZOblgut" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/js/bootstrap.min.js" integrity="sha384-B0UglyR+jN6CkvvICOB2joaf5I4l3gm9GU6Hc1og6Ls7i6U/mkkaduKaBhlAXv9k" crossorigin="anonymous"></script>
<!-- Font Awesome -->
<link href="https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
<!-- Moment.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.23.0/moment.min.js" integrity="sha256-VBLiveTKyUZMEzJd6z2mhfxIqz3ZATCuVMawPZGzIfA=" crossorigin="anonymous"></script>
<!-- Tempus Dominus Bootstrap 4 -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/tempusdominus-bootstrap-4/5.1.2/css/tempusdominus-bootstrap-4.min.css" integrity="sha256-XPTBwC3SBoWHSmKasAk01c08M6sIA5gF5+sRxqak2Qs=" crossorigin="anonymous" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/tempusdominus-bootstrap-4/5.1.2/js/tempusdominus-bootstrap-4.min.js" integrity="sha256-z0oKYg6xiLq3yJGsp/LsY9XykbweQlHl42jHv2XTBz4=" crossorigin="anonymous"></script>
</head>
<body>
<div class="input-group date" id="datetimepicker1" data-target-input="nearest">
<input type="text" class="form-control datetimepicker-input" data-target="#datetimepicker1"/>
<div class="input-group-append" data-target="#datetimepicker1" data-toggle="datetimepicker">
<div class="input-group-text"><i class="fa fa-calendar"></i></div>
</div>
</div>
<script>
$(function () {
$("#datetimepicker1").datetimepicker();
});
</script>
</body>
</html>
Direct Usage
The challenge now is to have this input snippet integrated with a Django form.
forms.py
from django import forms
class DateForm(forms.Form):
date = forms.DateTimeField(
input_formats=['%d/%m/%Y %H:%M'],
widget=forms.DateTimeInput(attrs={
'class': 'form-control datetimepicker-input',
'data-target': '#datetimepicker1'
})
)
template
<div class="input-group date" id="datetimepicker1" data-target-input="nearest">
{{ form.date }}
<div class="input-group-append" data-target="#datetimepicker1" data-toggle="datetimepicker">
<div class="input-group-text"><i class="fa fa-calendar"></i></div>
</div>
</div>
<script>
$(function () {
$("#datetimepicker1").datetimepicker({
format: 'DD/MM/YYYY HH:mm',
});
});
</script>
The script tag can be placed anywhere because the snippet $(function () { ... }); will run the datetimepicker initialization when the page is ready. The only requirement is that this script tag is placed after the jQuery script tag.
Custom Widget
You can create the widget in any app you want, here I'm going to consider we have a Django app named core.
core/widgets.py
from django.forms import DateTimeInput
class BootstrapDateTimePickerInput(DateTimeInput):
template_name = 'widgets/bootstrap_datetimepicker.html'
def get_context(self, name, value, attrs):
datetimepicker_id = 'datetimepicker_{name}'.format(name=name)
if attrs is None:
attrs = dict()
attrs['data-target'] = '#{id}'.format(id=datetimepicker_id)
attrs['class'] = 'form-control datetimepicker-input'
context = super().get_context(name, value, attrs)
context['widget']['datetimepicker_id'] = datetimepicker_id
return context
In the implementation above we generate a unique ID datetimepicker_id and also include it in the widget context.
Then the front-end implementation is done inside the widget HTML snippet.
widgets/bootstrap_datetimepicker.html
<div class="input-group date" id="{{ widget.datetimepicker_id }}" data-target-input="nearest">
{% include "django/forms/widgets/input.html" %}
<div class="input-group-append" data-target="#{{ widget.datetimepicker_id }}" data-toggle="datetimepicker">
<div class="input-group-text"><i class="fa fa-calendar"></i></div>
</div>
</div>
<script>
$(function () {
$("#{{ widget.datetimepicker_id }}").datetimepicker({
format: 'DD/MM/YYYY HH:mm',
});
});
</script>
Note how we make use of the built-in django/forms/widgets/input.html template.
Now the usage:
core/forms.py
from .widgets import BootstrapDateTimePickerInput
class DateForm(forms.Form):
date = forms.DateTimeField(
input_formats=['%d/%m/%Y %H:%M'],
widget=BootstrapDateTimePickerInput()
)
Now simply render the field:
template
{{ form.date }}
The good thing about having the widget is that your form could have several date fields using the widget and you could simply render the whole form like:
<form method="post">
{% csrf_token %}
{{ form.as_p }}
<input type="submit" value="Submit">
</form>
XDSoft DateTimePicker
The XDSoft DateTimePicker is a very versatile date picker and doesn't rely on moment.js or Bootstrap, although it looks good in a Bootstrap website.
It is easy to use and it is very straightforward.
You can download the source from GitHub releases page.
Below, a static example so you can see the minimum requirements and how all the pieces come together:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Static Example</title>
<!-- jQuery -->
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<!-- XDSoft DateTimePicker -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/jquery-datetimepicker/2.5.20/jquery.datetimepicker.min.css" integrity="sha256-DOS9W6NR+NFe1fUhEE0PGKY/fubbUCnOfTje2JMDw3Y=" crossorigin="anonymous" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery-datetimepicker/2.5.20/jquery.datetimepicker.full.min.js" integrity="sha256-FEqEelWI3WouFOo2VWP/uJfs1y8KJ++FLh2Lbqc8SJk=" crossorigin="anonymous"></script>
</head>
<body>
<input id="datetimepicker" type="text">
<script>
$(function () {
$("#datetimepicker").datetimepicker();
});
</script>
</body>
</html>
Direct Usage
A basic integration with Django would look like this:
forms.py
from django import forms
class DateForm(forms.Form):
date = forms.DateTimeField(input_formats=['%d/%m/%Y %H:%M'])
Simple form, default widget, nothing special.
Now using it on the template:
template
{{ form.date }}
<script>
$(function () {
$("#id_date").datetimepicker({
format: 'd/m/Y H:i',
});
});
</script>
The id_date is the default ID Django generates for the form fields (id_ + name).
Custom Widget
core/widgets.py
from django.forms import DateTimeInput
class XDSoftDateTimePickerInput(DateTimeInput):
template_name = 'widgets/xdsoft_datetimepicker.html'
widgets/xdsoft_datetimepicker.html
{% include "django/forms/widgets/input.html" %}
<script>
$(function () {
$("input[name='{{ widget.name }}']").datetimepicker({
format: 'd/m/Y H:i',
});
});
</script>
To have a more generic implementation, this time we are selecting the field to initialize the component using its name instead of its id, should the user change the id prefix.
Now the usage:
core/forms.py
from django import forms
from .widgets import XDSoftDateTimePickerInput
class DateForm(forms.Form):
date = forms.DateTimeField(
input_formats=['%d/%m/%Y %H:%M'],
widget=XDSoftDateTimePickerInput()
)
template
{{ form.date }}
Fengyuan Chen's Datepicker
This is a very beautiful and minimalist date picker. Unfortunately there is no time support. But if you only need dates this is a great choice.
To install this datepicker you can either use their CDN or download the sources from their GitHub releases page. Please note that they do not provide a compiled/processed JavaScript files. But you can download those to your local machine using the CDN.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Static Example</title>
<style>body {font-family: Arial, sans-serif;}</style>
<!-- jQuery -->
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<!-- Fengyuan Chen's Datepicker -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/datepicker/0.6.5/datepicker.min.css" integrity="sha256-b88RdwbRJEzRx95nCuuva+hO5ExvXXnpX+78h8DjyOE=" crossorigin="anonymous" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/datepicker/0.6.5/datepicker.min.js" integrity="sha256-/7FLTdzP6CfC1VBAj/rsp3Rinuuu9leMRGd354hvk0k=" crossorigin="anonymous"></script>
</head>
<body>
<input id="datepicker">
<script>
$(function () {
$("#datepicker").datepicker();
});
</script>
</body>
</html>
Direct Usage
A basic integration with Django (note that we are now using DateField instead of DateTimeField):
forms.py
from django import forms
class DateForm(forms.Form):
date = forms.DateTimeField(input_formats=['%d/%m/%Y %H:%M'])
template
{{ form.date }}
<script>
$(function () {
$("#id_date").datepicker({
format:'dd/mm/yyyy',
});
});
</script>
Custom Widget
core/widgets.py
from django.forms import DateInput
class FengyuanChenDatePickerInput(DateInput):
template_name = 'widgets/fengyuanchen_datepicker.html'
widgets/fengyuanchen_datepicker.html
{% include "django/forms/widgets/input.html" %}
<script>
$(function () {
$("input[name='{{ widget.name }}']").datepicker({
format:'dd/mm/yyyy',
});
});
</script>
Usage:
core/forms.py
from django import forms
from .widgets import FengyuanChenDatePickerInput
class DateForm(forms.Form):
date = forms.DateTimeField(
input_formats=['%d/%m/%Y %H:%M'],
widget=FengyuanChenDatePickerInput()
)
template
{{ form.date }}
Conclusions
The implementation is very similar no matter what date/datetime picker you are using. Hopefully this tutorial provided some insights on how to integrate this kind of frontend library to a Django project.
As always, the best source of information about each of those libraries are their official documentation.
I also created an example project to show the usage and implementation of the widgets for each of the libraries presented in this tutorial. Grab the source code at github.com/sibtc/django-datetimepicker-example.
03 Jan 2019 10:35pm GMT
Year in Review (2018)
An overview of 2018 and looking ahead to 2019.
03 Jan 2019 4:28pm GMT
Into 2019!
A new year has come around and it's time to both look back at the old and onward to the future of Evennia, the Python MUD creation system!
Last year
Last year saw the release of Evennia 0.8. This version of Evennia changes some fundamental aspects of the server infrastructure so that the server can truly run in daemon mode as you would expect (no more running it in a GnuScreen session if you want to see logging to the terminal). It also adds the new Online Creation System, which lets builders create and define prototypes using a menu system as well as big improvements in the web client, such as multiple window-panes (allows the user to assign text to different windows to keep their client uncluttered) as well as plenty of fixes and features to help ease life for the Evennia developer. Thanks again to everyone who helped out and contributed to the release of Evennia 0.8!
On a personal note, I spoke about Evennia at PyCon Sweden this December, which was fun. I might put up my talk and make a more detailed blog post about that in the future, but my talk got a surprising amount of attention and positive feedback. Clearly many people have fond memories of MUDs and enjoy seeing they are not only still around but are possible to create in Python!
This year
Now we are steaming ahead towards Evennia 0.9! Already we have had a large number of contributions towards this release. A coming change is a much improved central way to make typeclass information available to a website as well as central ways to do various common operations like creating characters and linking them to Accounts. Much of this functionality is today hidden in the various default commands. In Evennia 0.9 they will be moved into more easily-findable locations as library functions everyone can just trigger without having to copy functionality.
The biggest change for Evennia 0.9 is however that we will now finally move Evennia over to Python 3. This is now possible as all our dependencies are now sufficiently ported. As discussed in the community for a long time, this will be a clean break - we will not offer any mid-way solution of Python2/3 but will drop Python2 support entirely. We will at the same time move to Django 2.+. In Django 0.9 we will however probably not use any special Python3 features yet. It should hopefully not be too difficult for our users to upgrade, but we'll of course publish extensive instructions and help when that time comes.
We will likely support a minimum of Python 3.6 and maybe 3.7. This work is currently happening in a separate branch develop-py3 which will soonish merge into the current Python2.7-based develop branch and become Evennia 0.9 when it merges into master branch at an unknown time later this year.
There are a slew of other things planned for Evennia 0.9 and you can follow the progress from our project page. If you want to help out you are of course also very welcome. If you are new and are interested in getting your feet wet in helping out with Open-source development, we have a list of good first issues you could jump into.
Onward into the new year!
Fireworks image courtesy: U.S. Air Force photo by Zachary Hada
03 Jan 2019 1:22am GMT
02 Jan 2019
Django community aggregator: Community blog posts
Top 14 Pros of Using Django for Web Development
Django is one of the top frameworks for web development, but why is it so popular among developers and business owners? Let's review the reasons why so many applications and features are being developed with Django.
1. Django is simple
Django's documentation is exemplary. It was initially launched with high-quality docs, and they are still maintained at the same level, which makes it easy to use.
More than that, one of Django's main purposes is to simplify the development process: it covers the basics, so you can focus on the more unique and/or complex features of your project.
2. Django works on Python
The framework is based on Python - a high-level, dynamic, and interpreted programming language, well-loved by developers.
Although it's hard to find a language that can cover most programming tasks and problems, Python is a great choice for many of them. It's one of the most popular languages of 2018, competing with C/++ and Java.
Python is:
- Portable. Your code can be ported to many platforms, from PC and Linux to PlayStation.
- Multi-paradigm. It supports object-oriented programing, which is a simple way to code, as well as imperative programming.
- More interactive than most other languages. It resembles a pseudo-code language and helps you focus on solving a task, rather than on syntax.
Python web development with Django requires less code and less effort. Also, Python has extensive libraries, which make it easy to learn or switch to this language from another one. Customers like Python since it usually takes less time to write the code and, thus, less money to complete the technical part of a project.
3. Django has many useful features and extras to simplify development
Django has adopted Python's "batteries included" approach - the framework has everything necessary to develop a fully fledged application out of the box.
You don't need to spend hours customizing it to build a simple application or a prototype since all of the essentials are already available. But if you need additional features for a more complex app, there are well over 4,000 packages for Django to cover profiling, testing, and debugging.
The framework also has tool packages for working with cutting-edge technology such as data analysis, AI, and machine learning. They are easy to set up and use in your project. Plus, they're great if you're using Django for FinTech or other math-heavy industries.
4. Django is time-effective
Is Django good for web development of MVPs and prototypes? Yes, thanks to multiple features that make it time- and cost-effective.
Let's sum them up:
- There's a flexible, well-structured admin panel, better than Laravel or Yii's, for example.
- It allows you to reuse code from current or other projects (there is also a library of reusable apps, tools, and features).
- It has great templates and forms; they were even copied by other projects.
- It has many out-of-the-box libraries and tools that allow you to assemble a good prototype in record time.
5. Django suits any kind of project
Django is not an enterprise solution like C# or Java, yet it suits most types of projects, no matter their size. For example, if you're building a social media type web application, Django can handle the growth at any scale and capacity, be it heavy traffic or volumes of information. But if you want to make something simple, using Django for web development of a blog or a book database, for instance, is an excellent choice as well since it has everything you need to quickly assemble a working application.
In addition to that, Django is:
- Cross-platform. You can create applications that will run on Windows, as well as on Mac or Linux.
- Compatible with most major databases. You can use one or several different databases in one project thanks to Django's ORM, and switch between the databases with only one line of code.
6. Django is DRY and KISS compliant
Django follows the DRY (Don't Repeat Yourself) principle, which means you can replace frequently repeated software patterns with abstractions, or use data normalization. This way, you avoid redundancy and bugs. Plus, reusing the code simplifies development so you can focus on coding unique features.
KISS means "Keep It Short and Simple", among its many variations. In Django, it means simple, easy to read, and understandable code. For example, methods shouldn't be longer than 40-50 lines.
7. Django is secure and up-to-date
Django is always kept up to a high standard, following the latest trends in website security and development. That definitely answers the question "Is Django good for web development?" - as security is a priority in any project. Django is updated regularly with security patches, and even if you're using an older version of the framework, its security is still maintained with new patches. It's no wonder since Django has an LTS (Long-term Support) version.
8. Django is backward-compatible
You can use the interface of Django's older versions, and most of its features and formats. In addition, it has an understandable roadmap and descriptions - the release notes contain all the information you need to know about changes and, more importantly, when new changes become incompatible with previous releases.
9. Django is compatible with DevOps
You can also enhance your project using the DevOps methodology, which aims to shorten lifecycles while maintaining business objectives. It's especially good if you're using Django for banking web applications since they are quite complex.
It's great because you can:
- Solve problems faster with improved operational support.
- Use the continuous delivery approach (an app is produced in short cycles to ensure it is reliable enough to be released at any time).
- Increase the productivity of your team through collaborative working.
10. Django has its own infrastructure
Django doesn't depend on any outside solutions. It has pretty much everything, from a web server and a templating engine to an Object Relational Mapper (ORM), which allows the framework to use different databases and switch between them within one project.
Plus, Django has libraries and tools for building forms to receive input from users. That's important for any website that's supposed to do more than just publish content.
11. Django has a REST framework for building APIs
The benefits of using Django for web development also include its Representational State Transfer (REST) framework - a popular toolkit for building web APIs. Django's REST is powerful enough to build a ready-to-use API in just three lines of code.
One of its key advantages is that it's extremely flexible: data is not tied to any methods or resources, so REST can return different data formats and handle multiple types of calls. As a result, it can meet the requirements of different customers.
12. Django is time-tested
The Django framework has been around for more than a decade, and during that time, it has become the choice of many companies for creating their web applications.
A few of the famous examples are:
- YouTube;
- Instagram;
- Spotify;
- NASA;
- Disqus.
13. Django has a big, supportive, and professional community
Advantages of Django also include its big, professional community. It's quite easy to find good developers who know Django inside out and have experience coding with it.
That's a good testament to the framework's popularity - but it also means that:
- You can find help or, at least, the right direction in solving harder programming cases;
- The Django community is quick to respond to bugs and fix them;
- As an open source framework, Django is constantly improving - by means of new libraries, for example.
14. It's easy to find Django developers to hire
A huge advantage of the large Django community is that it's easy to find good developers for your team. Moreover, you can extend an existing team, since all Django developers use the same documentation, code pretty much the same way, and can easily read each other's code.
Bottom Line
The numerous advantages of Django framework can be summarized in three short phrases: less effort, less time, and less money.
You can use Django to start a small, simple project, and continue using it when the project grows, ensuring its high quality, functionality, and security. You can also use it to test an idea and save a lot of money if you find the project won't be worth investing in.
On the other hand, Django allows you to build a complex web application that can handle heavy traffic and huge volumes of information. It also has numerous packages with additional tools to power cutting-edge technology such as data analysis and machine learning.
Django could be the best fit for your next business idea regardless of what type of software project it is.
Your guide to the latest trends of product development
The post Top 14 Pros of Using Django for Web Development appeared first on Django Stars Blog.
02 Jan 2019 8:20pm GMT
How to Implement Grouped Model Choice Field
The Django forms API have two field types to work with multiple options: ChoiceField and ModelChoiceField.
Both use select input as the default widget and they work in a similar way, except that ModelChoiceField is designed to handle QuerySets and work with foreign key relationships.
A basic implementation using a ChoiceField would be:
class ExpenseForm(forms.Form):
CHOICES = (
(11, 'Credit Card'),
(12, 'Student Loans'),
(13, 'Taxes'),
(21, 'Books'),
(22, 'Games'),
(31, 'Groceries'),
(32, 'Restaurants'),
)
amount = forms.DecimalField()
date = forms.DateField()
category = forms.ChoiceField(choices=CHOICES)
Grouped Choice Field
You can also organize the choices in groups to generate the <optgroup> tags like this:
class ExpenseForm(forms.Form):
CHOICES = (
('Debt', (
(11, 'Credit Card'),
(12, 'Student Loans'),
(13, 'Taxes'),
)),
('Entertainment', (
(21, 'Books'),
(22, 'Games'),
)),
('Everyday', (
(31, 'Groceries'),
(32, 'Restaurants'),
)),
)
amount = forms.DecimalField()
date = forms.DateField()
category = forms.ChoiceField(choices=CHOICES)
Grouped Model Choice Field
When you are using a ModelChoiceField unfortunately there is no built-in solution.
Recently I found a nice solution on Django's ticket tracker, where someone proposed adding an opt_group argument to the ModelChoiceField.
While the discussion is still ongoing, Simon Charette proposed a really good solution.
Let's see how we can integrate it in our project.
First consider the following models:
models.py
from django.db import models
class Category(models.Model):
name = models.CharField(max_length=30)
parent = models.ForeignKey('Category', on_delete=models.CASCADE, null=True)
def __str__(self):
return self.name
class Expense(models.Model):
amount = models.DecimalField(max_digits=10, decimal_places=2)
date = models.DateField()
category = models.ForeignKey(Category, on_delete=models.CASCADE)
def __str__(self):
return self.amount
So now our category instead of being a regular choices field it is now a model and the Expense model have a relationship with it using a foreign key.
If we create a ModelForm using this model, the result will be very similar to our first example.
To simulate a grouped categories you will need the code below. First create a new module named fields.py:
fields.py
from functools import partial
from itertools import groupby
from operator import attrgetter
from django.forms.models import ModelChoiceIterator, ModelChoiceField
class GroupedModelChoiceIterator(ModelChoiceIterator):
def __init__(self, field, groupby):
self.groupby = groupby
super().__init__(field)
def __iter__(self):
if self.field.empty_label is not None:
yield ("", self.field.empty_label)
queryset = self.queryset
# Can't use iterator() when queryset uses prefetch_related()
if not queryset._prefetch_related_lookups:
queryset = queryset.iterator()
for group, objs in groupby(queryset, self.groupby):
yield (group, [self.choice(obj) for obj in objs])
class GroupedModelChoiceField(ModelChoiceField):
def __init__(self, *args, choices_groupby, **kwargs):
if isinstance(choices_groupby, str):
choices_groupby = attrgetter(choices_groupby)
elif not callable(choices_groupby):
raise TypeError('choices_groupby must either be a str or a callable accepting a single argument')
self.iterator = partial(GroupedModelChoiceIterator, groupby=choices_groupby)
super().__init__(*args, **kwargs)
And here is how you use it in your forms:
forms.py
from django import forms
from .fields import GroupedModelChoiceField
from .models import Category, Expense
class ExpenseForm(forms.ModelForm):
category = GroupedModelChoiceField(
queryset=Category.objects.exclude(parent=None),
choices_groupby='parent'
)
class Meta:
model = Expense
fields = ('amount', 'date', 'category')
Because in the example above I used a self-referencing relationship I had to add the exclude(parent=None) to hide the "group categories" from showing up in the select input as a valid option.
Further Reading
You can download the code used in this tutorial from GitHub: github.com/sibtc/django-grouped-choice-field-example
Credits to the solution Simon Charette on Django Ticket Track.
02 Jan 2019 3:51pm GMT
01 Jan 2019
Django community aggregator: Community blog posts
Caktus Blog: Top 18 Posts of 2018
In 2018, we published 44 posts on our blog, including technical how-to's, a series on UX research methods, web development best practices, and tips for project management. Among all those posts, 18 rose to the top of the popularity list in 2018.
Most Popular Posts of 2018
-
Creating Dynamic Forms with Django: Our most popular blog post delves into a straightforward approach to creating dynamic forms.
-
Make ALL Your Django Forms Better: This post also focuses on Django forms. Learn how to efficiently build consistent forms, across an entire website.
-
Django vs WordPress: How to decide?: Once you invest in a content management platform, the cost to switch later may be high. Learn about the differences between Django and WordPress, and see which one best fits your needs.
-
Basics of Django Rest Framework: Django Rest Framework is a library which helps you build flexible APIs for your project. Learn how to use it, with this intro post.
-
How to Fix your Python Code's Style: When you inherit code that doesn't follow your style preferences, fix it quickly with the instructions in this post.
-
Filtering and Pagination with Django: Learn to build a list page that allows filtering and pagination by enhancing Django with tools like django_filter.
-
Better Python Dependency Management with pip-tools: One of our developers looked into using pip-tools to improve his workflow around projects' Python dependencies. See what he learned with pip-tools version 2.0.2.
-
Types of UX Research: User-centered research is an important part of design and development. In this first post in the UX research series, we dive into the different types of research and when to use each one.
-
Outgrowing Sprints: A Shift from Scrum to Kanban: Caktus teams have used Scrum for over two years. See why one team decided to switch to Kanban, and the process they went through.
-
Avoiding the Blame Game in Scrum: The words we use, and the tone in which we use them, can either nurture or hinder the growth of Scrum teams. Learn about the importance of communicating without placing blame.
-
What is Software Quality Assurance?: A crucial but often overlooked aspect of software development is quality assurance. Find out more about its value and why it should be part of your development process.
-
Quick Tips: How to Find Your Project ID in JIRA Cloud: Have you ever created a filter in JIRA full of project names and returned to edit it, only to find all the project names replaced by five-digit numbers with no context? Learn how to find the project in both the old and new JIRA experience.
-
UX Research Methods 2: Analyzing Behavior: Learn about UX research methods best suited to understand user behavior and its causes.
-
UX Research Methods 3: Evaluating What Is: One set of techniques included in UX research involves evaluating the landscape and specific instances of existing user experience. Learn more about competitive landscape review.
-
Django or Drupal for Content Management: Which Fits your Needs?: If you're building or updating a website, you should integrate a content management system (CMS). See the pros and cons of Django and Drupal, and learn why we prefer Django.
-
5 Scrum Master Lessons Learned: Whether your team is new to Scrum or not, check out these lessons learned. Some are practical, some are abstract, and some are helpful reminders like "Stop being resistant to change, let yourself be flexible."
-
Add Value To Your Django Project With An API: This post for business users and beginning coders outlines what an API is and how it can add value to your web development project.
-
Caktus Blog: Best of 2017: How appropriate that the last post in this list is about our most popular posts from the previous year! So, when you've read the posts above, check out our best posts from 2017.
Thank You for Reading Our Blog
We look forward to giving you more content in 2019, and we welcome any questions, suggestions, or feedback. Simply leave a comment below.
01 Jan 2019 7:00pm GMT
23 Dec 2018
Django community aggregator: Community blog posts
How to Optimize Your Django REST Viewsets
The combination of Django and the Django REST framework is powerful. With just three classes (a model, a viewset, and a serializer), you can have a basic CRUD endpoint up and running. Although it is easy to get set up, it is also easy to end up with a view that makes hundreds of unnecessary database queries. As database queries are relatively slow, we want to avoid them as much as possible. In order to do this, we will follow Tip #4 from my Django ORM Optimization Tips post:
4. Use
select_related()andprefetch_related()when you will need foreign-key/reverse related objects.
Example
The example we will be working with is based on a blog site.
Here are the models:
from django.db import models
class BlogPost(models.Model):
title = models.CharField(max_length=100)
body = models.CharField(max_length=200)
author = models.ForeignKey(
'User', on_delete=models.CASCADE, related_name='posts')
def __str__(self):
return '{} - {}'.format(self.author.name, self.title)
class User(models.Model):
username = models.CharField(max_length=100)
name = models.CharField(max_length=100)
def __str__(self):
return '{} - {}'.format(self.username, self.name)
class Comment(models.Model):
author = models.ForeignKey(
'User', on_delete=models.CASCADE, related_name='comments')
comment = models.CharField(max_length=200)
post = models.ForeignKey(
'BlogPost', on_delete=models.CASCADE, related_name='comments')
def __str__(self):
return '{} - {}'.format(self.post, self.comment)
Here are the serializers:
from rest_framework import serializers
from .models import BlogPost, Comment, User
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = ('id', 'name', 'username',)
class CommentSerializer(serializers.ModelSerializer):
author = UserSerializer()
class Meta:
model = Comment
fields = ('id', 'author', 'comment', 'post')
class BlogPostSerializer(serializers.ModelSerializer):
author = UserSerializer()
comments = CommentSerializer(many=True)
class Meta:
model = BlogPost
fields = ('id', 'author', 'body', 'comments', 'title',)
Here is the viewset:
from rest_framework import viewsets
from .models import BlogPost
from .serializers import BlogPostSerializer
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = BlogPost.objects.all()
serializer_class = BlogPostSerializer
And finally, an example response from our blog posts endpoint looks like this:
[
{
"id": 1,
"author": {
"id": 1,
"name": "Dalinar",
"username": "user1"
},
"body": ".........",
"comments": [
{
"id": 1,
"author": {
"id": 2,
"name": "Kaladin",
"username": "user2"
},
"comment": ".........",
"post": 1
},
{
"id": 2,
"author": {
"id": 3,
"name": "Shallan",
"username": "user3"
},
"comment": ".........",
"post": 1
},
{
"id": 3,
"author": {
"id": 1,
"name": "Dalinar",
"username": "user1"
},
"comment": ".........",
"post": 1
}
],
"title": "Dalinar's Blog Post"
},
{
"id": 2,
"author": {
"id": 3,
"name": "Shallan",
"username": "user3"
},
"body": ".........",
"comments": [
{
"id": 4,
"author": {
"id": 2,
"name": "Kaladin",
"username": "user2"
},
"comment": ".........",
"post": 2
},
{
"id": 5,
"author": {
"id": 1,
"name": "Dalinar",
"username": "user1"
},
"comment": ".........",
"post": 2
}
],
"title": "Shallan's Blog Post"
}
]
How to Profile
In order to optimize anything, it is important to know how to profile it. There are many different ways to profile Django views, but I like to start out by creating a print statement with the number of database queries that have happened so far.
from rest_framework import viewsets
from .models import BlogPost
from .serializers import BlogPostSerializer
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = BlogPost.objects.all()
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
print('# of Queries: {}'.format(len(connection.queries)))
return response
Here, we have overridden the dispatch() method, which is the entry point for all Django views. The print statement prints the number of database queries that have happened so far, so you want to place it after the code you are profiling. You can even place one before and after a block of code and take the difference in order to narrow down the culprit.
Sending a request to /blog-posts/ produces the following in the console:
# of Queries: 10 [22/Dec/2018 17:53:45] "GET /blog-posts/ HTTP/1.1" 200 11732
It currently takes ten database queries to serialize two blog posts. We can improve on this.
Use select_related() for foreign key relationships
The thing we need to look for is foreign key relationships that are being serialized. Let's take another look at the BlogPost model and serializer.
class BlogPost(models.Model):
title = models.CharField(max_length=100)
body = models.CharField(max_length=200)
author = models.ForeignKey(
'User', on_delete=models.CASCADE, related_name='posts')
def __str__(self):
return '{} - {}'.format(self.author.name, self.title)
class BlogPostSerializer(serializers.ModelSerializer):
author = UserSerializer()
comments = CommentSerializer(many=True)
class Meta:
model = BlogPost
fields = ('id', 'author', 'body', 'comments', 'title',)
Now we can see that BlogPost has a foreign key relationship author to User, and it has been included in the serializer. By default, Django will make a database query each time an author needs to be serialized. To avoid this, we can use select_related() on the viewset's queryset to tell Django to merge this extra query with the primary BlogPost query.
from rest_framework import viewsets
from .models import BlogPost
from .serializers import BlogPostSerializer
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = (
BlogPost.objects
.select_related(
'author',
)
)
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
print('# of Queries: {}'.format(len(connection.queries)))
return response
Sending another response to our endpoint produces this:
# of Queries: 8 [22/Dec/2018 17:57:11] "GET /blog-posts/ HTTP/1.1" 200 11732
We only reduced the number of queries by one per blog post. We still have some work to do.
Use prefetch_related() for reverse relationships
The next thing to look out for is reverse relationships that are being serialized. Let's take a look at the serializer again, and the Comment model this time.
class BlogPostSerializer(serializers.ModelSerializer):
author = UserSerializer()
comments = CommentSerializer(many=True)
class Meta:
model = BlogPost
fields = ('id', 'author', 'body', 'comments', 'title',)
class Comment(models.Model):
author = models.ForeignKey(
'User', on_delete=models.CASCADE, related_name='comments')
comment = models.CharField(max_length=200)
post = models.ForeignKey(
'BlogPost', on_delete=models.CASCADE, related_name='comments')
def __str__(self):
return '{} - {}'.format(self.post, self.comment)
Here we see that BlogPostSerializer serializes a list of comments from the reverse relationship to Comment. Without optimization, this will cause a database query to occur for each blog post. We can use prefetch_related() on the viewset's queryset to reduce this to just one extra query (in addition to the primary BlogPost query) to get this data for all blog posts.
from rest_framework import viewsets
from .models import BlogPost
from .serializers import BlogPostSerializer
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = (
BlogPost.objects
.select_related(
'author',
)
.prefetch_related(
'comments',
)
)
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
print('# of Queries: {}'.format(len(connection.queries)))
return response
Sending another request to our endpoint produces:
# of Queries: 7 [22/Dec/2018 18:02:36] "GET /blog-posts/ HTTP/1.1" 200 11732
This reduced our total numbers of queries down by one, as expected, but there are still a lot of queries happening. We need to go one level deeper and look at CommentSerializer.
class CommentSerializer(serializers.ModelSerializer):
author = UserSerializer()
class Meta:
model = Comment
fields = ('id', 'author', 'comment', 'post')
Aha! Comment has a foreign key relationship author to User that it is serializing. We need to include this in prefetch_related(), like so:
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = (
BlogPost.objects
.select_related(
'author',
)
.prefetch_related(
'comments__author',
)
)
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
print('# of Queries: {}'.format(len(connection.queries)))
return response
Now, hitting our endpoint again produces this:
# of Queries: 3 [22/Dec/2018 18:07:53] "GET /blog-posts/ HTTP/1.1" 200 11732
That resulted in a large improvement. However, for a single queryset, we should generally shoot for one primary query and one extra query per reverse relationship. Since we only have one reverse relationship here, we should be shooting for no more than two database queries. We need to dig a little deeper.
Use Prefetch objects to control prefetch_related() at a deeper level
When we get down to a smaller number of queries, it is sometimes beneficial to print out the actual queries to see where they are coming from. We can do this by printing out connection.queries instead of len(connection.queries).
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = (
BlogPost.objects
.select_related(
'author',
)
.prefetch_related(
'comments__author',
)
)
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
for query in connection.queries:
print(query['sql'])
return response
This yields the following when hitting our endpoint:
SELECT "demonstration_blogpost"."id", "demonstration_blogpost"."title", "demonstration_blogpost"."body", "demonstration_blogpost"."author_id", "demonstration_user"."id", "demonstration_user"."username", "demonstration_user"."name" FROM "demonstration_blogpost" INNER JOIN "demonstration_user" ON ("demonstration_blogpost"."author_id" = "demonstration_user"."id")
SELECT "demonstration_comment"."id", "demonstration_comment"."author_id", "demonstration_comment"."comment", "demonstration_comment"."post_id" FROM "demonstration_comment" WHERE "demonstration_comment"."post_id" IN (1, 2)
SELECT "demonstration_user"."id", "demonstration_user"."username", "demonstration_user"."name" FROM "demonstration_user" WHERE "demonstration_user"."id" IN (1, 2, 3)
[23/Dec/2018 16:09:35] "GET /blog-posts/ HTTP/1.1" 200 11732
The first thing to look at when looking at these queries is the FROM clause. This tells you which table is being queried, and usually directly corresponds to one of your models. The first query is FROM demonstration_blogpost, so we know this is the primary BlogPost query we expect. The second one is FROM demonstration_comment, so this is the query on Comment that we expect of a reverse relationship. The final one, however, is FROM demonstration_user. This is the one we need to try to eliminate.
This case is actually hard to figure out without knowing the details of how Django handles foreign key relationships on reverse relationships. You can read the docs for more information, but the gist is that adding comments__author to our prefetch_related() results in a query on BlogPost, Comment, and User. In order to prevent this, we need to tell Django to include the author data in the Comment query. Django provides a Prefetch object that allows this extra level of control.
from django.db.models import Prefetch
from rest_framework import viewsets
from .models import BlogPost, Comment
from .serializers import BlogPostSerializer
class BlogPostViewSet(viewsets.ModelViewSet):
queryset = (
BlogPost.objects
.select_related(
'author',
)
.prefetch_related(
Prefetch(
'comments',
queryset=Comment.objects.select_related('author')
)
)
)
serializer_class = BlogPostSerializer
def dispatch(self, *args, **kwargs):
response = super().dispatch(*args, **kwargs)
# For debugging purposes only.
from django.db import connection
print('# of Queries: {}'.format(len(connection.queries)))
return response
Using Prefetch allows us to define the queryset for the prefetch. This means we can use select_related() to merge the comments and comments__author queries into one. Let's try hitting our endpoint again.
# of Queries: 2 [23/Dec/2018 16:34:21] "GET /blog-posts/ HTTP/1.1" 200 11732
Finally, we're down to two queries. This is the best we can do, as each prefetch_related() argument causes at least one database query.
Closing Remarks
Although our example started with ten database queries, each blog post added to the system would have increased that number. That can have a crippling effect on your system as your data grows in volume. After optimization, however, the database queries required to retrieve our data will always be two; no matter how much data accumulates.
This is a common theme when working with Django. Interacting with the database is easy and concise, but you can end up with something really inefficient if you don't know what to look out for. I highly recommend reading my Django ORM Optimization Tips post to become more effective with the Django ORM. Also see Normalize Your Django REST Serializers to make your data easier for your client to use.
For full example code, check out the GitHub repository.
The post How to Optimize Your Django REST Viewsets appeared first on concise coder.
23 Dec 2018 11:48pm GMT