fidzu : django

28 May 2020 5:31pm GMT

In this episode, I worked on a views and templates. There are a number of core pages that are required to flesh out the minimal interface for the app. We're building them. I began by showing the page that we were going to work on. I outlined the changes I planned to make, then we started. The first thing we added was data about the school year, the main model on display in the page.

28 May 2020 5:00am GMT

One of my colleagues has spent a bunch of time investigating and then implementing some testing using [Robot Framework](https://robotframework.com). Whilst at times the command line feels like it was written by someone who hasn't used unix much, it's pretty powerful. There are also some nice tools, like several Google Chrome [plugins](https://chrome.google.com/webstore/detail/robotcorder/ifiilbfgcemdapeibjfohnfpfmfblmpd) [that](https://chrome.google.com/webstore/detail/chrome-robot/dihdbpkpgdkioobahfpnkondnekhbmlo) will record what you are doing and generate a script based upon that. There are also other tools to [help build testing scripts](https://chrome.google.com/webstore/detail/page-modeller-selenium-ro/ejgkdhekcepfgdghejpkmbfjgnioejak). There is also an existing [DjangoLibrary](https://pypi.org/project/robotframework-djangolibrary/) for integrating with [Django](https://www.djangoproject.com/). It's an interesting approach: you install some extra middlewary that allows you to perform requests directly to the server to create instances using [Factory Boy](https://factoryboy.readthedocs.io/), or fetch data from Querysets. However, it requires that the data is serialised before sending to the django server, and the same the other way. This means, for instance, that you cannot follow object references to get a related object without a bunch of legwork: usually you end up doing another `Query Set` query. There are some things in it that I do not like: * A new instance of the django `runserver` command is started for each Test Suite. In our case, this takes over 10 seconds to start as all imports are processed. * The database is flushed between Test Suites. We have data that is added through migrations that is required for the system to operate correctly, and in some cases for tests to execute. This is the same problem I've seen with `TransactionTestCase`. * Migrations are applied before running each Test Suite. This is unnecessary, and just takes more time. * Migrations are created automatically before running each Test Suite. This is just the wrong approach: at worst you'd want to warn that migrations are not up to date - otherwise you are testing migrations that may not have been committed: your CI would pass because the migrations were generated, but your system would fail in reality because those migrations do not really exist. Unless you are also making migrations directly on your production server and not committing them at all, in which case you really should stop that. That's in addition to having to install extra middleware. But, back onto the initial issue: interacting with Django models. What would be much nicer is if you could just call the python code directly. You'd get python objects back, which means you can follow references, and not have to deal with serialisation. It's fairly easy to write a Library for Robot Framework, as it already runs under Python. The tricky bit is that to access Django models (or Factory Boy factories), you'll want to have the Django infrastructure all managed for you. Let's look at what the `DjangoLibrary` might look like if you are able to assume that `django` is already available and configured: {% highlight python %} import importlib from django.apps import apps from django.core.urlresolvers import reverse from robot.libraries.BuiltIn import BuiltIn class DjangoLibrary: """ Tools for making interaction with Django easier. Installation: ensure that in your `resource.robot` or test file, you have the following in your "***Settings***" section: Library djangobot.DjangoLibrary ${HOSTNAME} ${PORT} The following keywords are provided: Factory: execute the named factory with the args and kwargs. You may omit the 'factories' module from the path to reduce the amount of code required. ${obj}= Factory app_label.FactoryName arg kwarg=value ${obj}= Factory app_label.factories.FactoryName arg kwarg=value Queryset: return a queryset of the installed model, using the default manager and filtering according to any keyword arguments. ${qs}= Queryset auth.User pk=1 Method Call: Execute the callable with tha args/kwargs provided. This differs from the Builtin "Call Method" in that it expects a callable, rather than an instance and a method name. ${x}= Method Call ${foo.bar} arg kwargs=value Relative Url: Resolve the named url and args/kwargs, and return the path. Not quite as useful as the "Url", since it has no hostname, but may be useful when dealing with `?next=/path/` values, for instance. ${url}= Relative Url foo:bar baz=qux Url: Resolve the named url with args/kwargs, and return the fully qualified url. ${url}= Url foo:bar baz=qux Fetch Url: Resolve the named url with args/kwargs, and then using SeleniumLibrary, navigate to that URL. This should be used instead of the "Go To" command, as it allows using named urls instead of manually specifying urls. Fetch Url foo:bar baz=qux Url Should Match: Assert that the current page matches the named url with args/kwargs. Url Should Match foo:bar baz=qux """ def __init__(self, hostname, port, **kwargs): self.hostname = hostname self.port = port self.protocol = kwargs.pop('protocol', 'http') @property def selenium(self): return BuiltIn().get_library_instance('SeleniumLibrary') def factory(self, factory, **kwargs): module, name = factory.rsplit('.', 1) factory = getattr(importlib.import_module(module), name) return factory(**kwargs) def queryset(self, dotted_path, **kwargs): return apps.get_model(dotted_path.split('.'))._default_manager.filter(**kwargs) def method_call(self, method, *args, **kwargs): return method(*args, **kwargs) def fetch_url(self, name, *args, **kwargs): return self.selenium.go_to(self.url(name, *args, **kwargs)) def relative_url(self, name, *args, **kwargs): return reverse(name, args=args, kwargs=kwargs) def url(self, name, *args, **kwargs): return '{}://{}:{}'.format( self.protocol, self.hostname, self.port, ) + reverse(name, args=args, kwargs=kwargs) def url_should_match(self, name, *args, **kwargs): self.selenium.location_should_be(self.url(name, *args, **kwargs)) {% endhighlight %} You can write a management command: this allows you to hook in to Django's existing infrastructure. Then, instead of calling robot directly, you use `./manage.py robot` What's even nicer about using a management command is that you can have that (optionally, because in development you probably will already have a devserver running) start `runserver`, and kill it when it's finished. This is the same philosophy as `robotframework-DjangoLibrary` already does, but we can start it once before running out tests, and kill it at the end. So, what could our management command look like? Omitting the code for starting `runserver`, it's quite neat: {% highlight python %} from __future__ import absolute_import from django.core.management import BaseCommand, CommandError import robot class Command(BaseCommand): def add_arguments(self, parser): parser.add_argument('tests', nargs='?', action='append') parser.add_argument('--variable', action='append') parser.add_argument('--include', action='append') def handle(self, **options): robot_options = { 'outputdir': 'robot_results', 'variable': options.get('variable') or [] } if options.get('include'): robot_options['include'] = options['include'] args = [ 'robot_tests/{}_test.robot'.format(arg) for arg in options['tests'] or () if arg ] or ['robot_tests'] result = robot.run(*args, **robot_options) if result: raise CommandError('Robot tests failed: {}'.format(result)) {% endhighlight %} I think I'd like to do a bit more work on finding tests, but this works as a starting point. We can call this like: ./manage.py robot foo --variable BROWSER:firefox --variable PORT:8000 This will find a test called `robot_tests/foo_test.robot`, and execute that. If you omit the `test` argument, it will run on all tests in the `robot_tests/` directory. I've still got a bit to do on cleaning up the code that starts/stops the server, but I think this is useful even without that.

27 May 2020 10:34pm GMT