26 Aug 2016

feedPlanet Plone

Andreas Jung: XML-Director 2.0 update

XML-Director is our Plone-based integration platform for building high-quality enterprise-grade publishing solutions.

26 Aug 2016 5:18pm GMT

20 Aug 2016

feedPlanet Plone

Vikas Parashar: Google Summer of Code 2016 — Wrap Up

Welcome to the last GSoC report.

Continue reading on »

20 Aug 2016 8:52pm GMT

17 Aug 2016

feedPlanet Plone

Encolpe: Try-except versus if-else in Python 3.5

Today I ran again in the question why to use if-else when try-except is shorter?

There is a semantic part that we loose information on a try-except as we don't know what the developer did expect in the most case and there is a performance part. I found a test for python 2 (Try / Except Performance in Python: A Simple Test) with simple assumptions: we try to get a value in a one keyed dictionary and make iterations to get a statistic on this access. I made it a little more complex with a dictionary with 10,001 keys.

Results analysis:

Then the if-else has a predictable comportment and help the maintainer.

I'm agree that the semantic part of the if-else can be replace by a comment but i still not recommend to try-except if it is not to raise an exception.

The results:

The case where the key does not exist: 
 
1,000 iterations: 
with_try (0.250 ms) 
with_try_exc (0.291 ms) 
without_try (0.119 ms) 
without_try_not (0.120 ms) 
 
1,000,000 iterations: 
with_try (231.647 ms) 
with_try_exc (263.633 ms) 
without_try (119.238 ms) 
without_try_not (118.602 ms) 
 
1,000,000,000 iterations: 
with_try (224659.381 ms) 
with_try_exc (260333.897 ms) 
without_try (109796.531 ms) 
without_try_not (111871.690 ms) 
 
 
The case where the key does exist: 
 
1,000 iterations: 
exists_with_try (0.066 ms) 
exists_with_try_exc (0.070 ms) 
exists_without_try (0.166 ms) 
exists_without_try_not (0.180 ms) 
 
1,000,000 iterations: 
exists_with_try (57.661 ms) 
exists_with_try_exc (56.909 ms) 
exists_without_try (113.633 ms) 
exists_without_try_not (116.340 ms) 
 
1,000,000,000 iterations: 
exists_with_try (57650.440 ms) 
exists_with_try_exc (57395.376 ms) 
exists_without_try (114659.023 ms) 
exists_without_try_not (117646.034 ms)

The code:

#!/usr/bin/env python3

import time

d = dict.fromkeys(range(0, 10000), 0)
d['somekey'] = 1

ITERATIONS = (1000, 1000*1000, 1000*1000*1000)

def time_me(function):
    def wrap(*arg):
        start = time.time()
        r = function(*arg)
        end = time.time()
        print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
        return r
    return wrap


# Not Existing
@time_me
def with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except:
            pass

@time_me
def with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['notexist']
        except Exception:
            pass

@time_me
def without_try(iterations):
    for i in range(0, iterations):
        if d.get('notexist'):
            pass
        else:
            pass

@time_me
def without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('notexist'):
            pass
        else:
            pass



# Existing
@time_me
def exists_with_try(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except:
            pass

@time_me
def exists_with_try_exc(iterations):
    for i in range(0, iterations):
        try:
            get = d['somekey']
        except Exception:
            pass

@time_me
def exists_without_try(iterations):
    for i in range(0, iterations):
        if d.get('somekey'):
            pass
        else:
            pass

@time_me
def exists_without_try_not(iterations):
    for i in range(0, iterations):
        if not d.get('somekey'):
            pass
        else:
            pass


print ("\n\nThe case where the key does not exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    with_try(iteration)
    with_try_exc(iteration)
    without_try(iteration)
    without_try_not(iteration)


print ("\n\nThe case where the key does exist:")
for iteration in ITERATIONS:
    print ("\n%d iterations:" % iteration)
    exists_with_try(iteration)
    exists_with_try_exc(iteration)
    exists_without_try(iteration)
    exists_without_try_not(iteration)


17 Aug 2016 9:34am GMT

Encolpe: When UnicodeDecodeError become irrational check $LANG

I spent hours this week trying to understand how an installation script can fail on some installations.

In input we have an utf-8 encoded file and we add some xml files, also 'utf-8' encoded. These are parsed with Markdown.

python -m lom2mlr.markdown -l -c rationale.md

It is really simple but sometimes we ran into a strange error:

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/edegoute/Projects/lom2mlr/lom2mlr/lom2mlr/markdown/__main__.py", line 3, in <module>
compile()
File "lom2mlr/markdown/__init__.py", line 55, in compile
extensions=extensions)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 529, in markdownFromFile
kwargs.get('encoding', None))
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 441, in convertFile
html = self.convert(text)
File "/home/edegoute/Projects/lom2mlr/local/lib/python2.7/site-packages/markdown/__init__.py", line 375, in convert
newRoot = treeprocessor.run(root)
File "lom2mlr/markdown/test_mlr.py", line 76, in run
print(" " * int(element.tag[1]) + element.text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 4: ordinal not in range(128)

First, it was difficult to understand how an unicode error can raise an iso-8859-1 problem on utf-8 files. Going deep I found some known problems with 'codecs.open' in python2.7 but no solution. I tried to force Markown to treat these files as 'iso-8859-1' files, then it ran an utf-8 unicode error at the same line not in the opening. It was sounding too many magic for me.

At that point, I checked again the installation was identical: same python version, same pip version, same eggs versions. I tried some egg upgrade without any success. And finally came the idea to check environment variables. Bingo! On all systems with failing installations we have no localization ($LANG=C). The fix was so simple:

export LANG=en_US.UTF-8

That's it!

I still don't understand the magic in the codecs python module. Why it computes a different encoding when the function call already asks for one? The workaround is simple for programmers.


17 Aug 2016 9:19am GMT

12 Aug 2016

feedPlanet Plone

Ross Patterson: PyCon 2012 Talks

A bit about my experience at my first PyCon and the talks.

12 Aug 2016 5:33pm GMT

01 Aug 2016

feedPlanet Plone

Makina Corpus: The world's simplest Python template engine

A template engine is a component able to render some data using a given string template.

We use it extensively in web development (that's not surprising because web development is mainly about reading some data and render them as strings, that's what I say when someone ask me about my job).

So there are already a lot of Python based template engines (Jinja2, Mako, Chameleon, ...).

The purpose of this article is to build a very basic template engine based only on the Python standard string class formatting capacities.

You probably know...

You probably know .format() replaces the old "%" based formatting solution:

>>> "My name is %s and I am %d" % ("Eric", 41)
'My name is Eric and I am 41'
>>> "My name is {name} and I am {age}".format(name="Eric", age=41)
'My name is Eric and I am 41'

It allows to perform all the typical formatting features, for instance:

>>> '{:^20}'.format('centered')
' centered '
>>> '{:^20}'.format('align right')
' align right '
>>> '{:>20}'.format('align right')
' align right'
>>> '{:<20}'.format('align left')
'align left '

and many other things (have a look to https://pyformat.info/).

Did you know?

format() is also able to access attributes or items of the parameters.

For instance with a dictionary, we can do this:

>>> 'Name: {person[name]}, age: {person[age]}'.format(person={'name': 'Eric', 'age': 41})
'Name: Eric, age: 41'

And the same goes with attributes:

>>> class Song(object):
... title = 'Where is my mind'
...
>>> 'My favorite song is: {song.title}'.format(song=Song())
'My favorite song is: Where is my mind'

That's really cool. It starts looking like a template engine, right?

Ok, but few things are missing

What we usually expect from a template engine is to be able to:

Let's see how we can handle that.

Calling methods

A method is an attribute of an object, we can call attribute, why couldn't wecall a method? Let's try:

>>> 'My name is {name.upper}'.format(name='eric')
'My name is <built-in method upper of str object at 0x7f67dc8d1630>'

Yeah, not exactly what we expected...

An interesting feature of format() is the format specification: instead of just inserting a field with {field}, we can specify a format like this: {field:spec}.

That's exactly what we do with float for instance:

>>> "{:.3}".format(3.14159)
'3.14'

Well, it is actually very easy to implement our own spec by derivating the Formatter class. So let's implement a ':call' spec in charge of calling the current field:

class SuperFormatter(string.Formatter):

    def format_field(self, value, spec):
        if spec == 'call':
            return value()
        else:
            return super(SuperFormatter, self).format_field(value, spec)

We can use it that way:

>>> sf.format('My name is {name.upper:call}', name="eric")
'My name is ERIC'

Nice!

Loops

Similarly, we can implement a :repeat spec,

class SuperFormatter(string.Formatter):

    def format_field(self, value, spec):
        if spec.startswith('repeat'):
            template = spec.partition(':')[-1]
            if type(value) is dict:
                value = value.items()
            return ''.join([template.format(item=item) for item in value])
        else:
            return super(SuperFormatter, self).format_field(value, spec)

Here, we pass a parameter to the spec to provide the template to use when we iterate on the loop items. So the resulting format is: <field>:repeat:<template>.

This subtemplate is a regular format() template where we escape the curly brackets by doubling them, and where the only field is the loop variable (named item).

So we can use it like this:

>>> sf.format('''Table of contents:
... {chapters:repeat:Chapter {{item}}
... }''', chapters=["I", "II", "III", "IV"])
'''Table of contents:
Chapter I
Chapter II
Chapter III
Chapter IV
'''

Condition

Let's also implement a :if spec to test the field value, and then display or not the subtemplate:

class SuperFormatter(string.Formatter):

    def format_field(self, value, spec):
        if spec.startswith('if'):
            return (value and spec.partition(':')[-1]) or ''
        else:
            return super(SuperFormatter, self).format_field(value, spec)

At first, it seems stupid, because it looks like it will only be able to conditionnally display static portions, like this:

>>> sf.format('Action: Back / Logout {manager:if:/ Delete}', manager=False)
'Action: Back / Logout '
>>> sf.format('Action: Back / Logout {manager:if:/ Delete}', manager=True)
'Action: Back / Logout / Delete'

What if we want to render conditionnally a portion of template containing fields, like this:

>>> sf.format('Action: Back / Logout {manager:if:/ Delete {id}}', manager=False, id=34)
'Action: Back / Logout '
>>> sf.format('Action: Back / Logout {manager:if:/ Delete {id}}', manager=True, id=34)
'Action: Back / Logout / Delete 34'

Hey that works! That was unexpected. To be honest, I first wrote the test exactly like this, and I expected it to fail, but it was not! Why is that?

Because here the curly brackets are not escaped, so they are processed by the main format() call. So cool!

Here we go

That's it, we have a pretty nice and complete template engine.

The full code of our world's simplest Python template engine is here, the implementation itself is 10 lines long, Python is a really powerful language!

If you have any funny idea to improve it (remember, I want to keep it short), pull requests are welcome.

The objective is mostly to have fun with Python and demonstrate what the standard Formatter class is able to do, but, having a second though, I might intregate it in Rapido :).

01 Aug 2016 12:00am GMT

Makina Corpus: Paypal tracking with Rapido

Paypal Instant Payment Notification (IPN)

If we put a Paypal button somewhere in a website, it is nice to know if the user got through the payment process properly and did not just canceled in the middle.

To achieve this, Paypal provides an asynchronous messaging service named Instant Payment Notification (IPN). Here is the process:

How to do it with Rapido

Creating the Paypal button is easy, we just copy/paste the core provided by Paypal in a block (see the Rapido documentation to learn how to create blocks) and we add our "custom" variable:

pay.yaml:

elements:
trackingid:
type: BASIC

pay.py:

def trackingid(context):
currentUser = context.api.user.get_current()
return currentUser.getUserName() # or anything else relevant

pay.html:

<form action="https://www.sandbox.paypal.com/cgi-bin/webscr" method="post" target="_top">
<input type="hidden" name="custom" value="{trackingid}"/>
<input type="hidden" name="cmd" value="_s-xclick">
<input type="hidden" name="hosted_button_id" value="XXXXXXXXXX">
<input type="image" src="https://www.sandbox.paypal.com/fr_FR/FR/i/btn/btn_paynowCC_LG.gif" border="0" name="submit" alt="PayPal, le réflexe sécurité pour payer en ligne">
<img alt="" border="0" src="https://www.sandbox.paypal.com/fr_FR/i/scr/pixel.gif" width="1" height="1">
</form>

Note: make sure we do not insert the full block form but just its children with our Diazo rule because we would get a form into a form, so the rule will be like that:

<after css:content="h1.documentFirstHeading">
<include href="@@rapido/myapp/block/pay" css:content-children="form[name='pay']"/>
</after>

Now let's see how to implement the listener that will get the Paypal notification. At some point, we will need to create a POST requests, so we need an extra library for that. We will use the well-known Python requests library, so we need to declare it safe for Rapido.

Hence we add the following somewhere in our custom theme module, for instance in __init__.py (or anywhere else):

import requests
from rapido.core import app
app.safe_modules.requests = requests

Warning: allowing to use a module like Requests in Rapido can have consequences regarding security, so if you do not control who is allowed to create Rapido apps (by default, only managers), do not do it.

And now, we create a block named ipn, containing an element named listener, the IPN URL we will declare to Paypal will be:

http://my.server.com/@@rapido/myapp/block/ipn/listener

Here is the implementation:

ipn.yaml:

elements:
listener:
type: BASIC

ipn.py:

def listener(context):
trackingid = context.request.form['custom']
paypal_params = {'cmd': '_notify-validate'}
# collect all the Paypal params
for key in context.request.form:
paypal_params[key] = context.request.form[key]
# acknowledge with a POST
req = context.modules.requests.post(
'https://www.sandbox.paypal.com/cgi-bin/webscr',
data=paypal_params)
# check if everthing is ok
if req.text == "VERIFIED":
mark_payment_has_done(trackingid)
# return an empty answer to the Paypal original request
return ''

That's it, that's just 10 lines of code (of course we will need to implement mark_payment_has_done() depending on our case inb order to persist in a record that the payment is done and verified).

01 Aug 2016 12:00am GMT

29 Jul 2016

feedPlanet Plone

Andreas Jung: Stellungnahme der Antidiskriminierungsstelle des Bundes zur Beförderung von großen Personen bei Airlines

In einer formlosen Anfrage an die Antidiskriminierungsstelle des Bundes habe ich gefragt, ob es das Zusammenpferchen von großen Personen wie mich (203cm) in der Economy Klasse der meisten Airlines auf dem Platz einer Legehenne rechtlich vereinbar ist und ob sich aus der Körpergrösse ein Anrecht auf eine menschenwürdige Beförderung ableiten lässt. Die Antidiskriminierungsstelle antwortet mit einer interessanten Rechtseinschätzung....

29 Jul 2016 8:36am GMT

28 Jul 2016

feedPlanet Plone

Andreas Jung: Cross-browser hyphenation support for Plone 5

Most browsers lack hyphenation support (except Firefox). This Plone 5 add-on brings customizable hyphenation support to Plone 5 e.g. for better readability of the Plone 5 toolbar in German language.

28 Jul 2016 5:09am GMT

26 Jul 2016

feedPlanet Plone

Wildcard Corp: 5 Reasons Your Website Needs an Update

Having a website is no longer simply a flashy accessory for a business or organization; it's an absolute necessity. But a successful website is not something you can set up and forget. Website trends are extremely fluid, and what works for you now will not work a year from now. Visitors to your site will absolutely notice if you don't keep up with the times, and it will definitely have a negative impact on their impression of your organization or business. We have created a countdown of five ways you can tell that your website needs an update and how to fix it.

5: Your Website is Outdated

If your website looks like you haven't updated it in years, visitors will be instantly turned off. A negative first impression is difficult to shake in any context, but it's certainly true of the online world. If you want people to care about your website, your content, and your products, it has to look like you care about your website.

Since website trends change like the weather, sometimes it helps to get a professional perspective. Web developers know what's "in" right now and how to set it up. They can give you a set of tools that you can use to make your website really stand out. Many developers have designers on staff that can make your website look exactly the way you always imagined.

4: Your Website is Hard to Find

When someone does an internet search, they may not know your website by name. You could be offering the deal of a lifetime, but if they can't find it, they won't buy it. For example, if you Google the main service you provide and your location (i.e. the name of your city) and your site doesn't show up on the first page of results, that should be a red flag that you need to overhaul not only your site's keywords, but your site's content in general.

Through a process called search engine optimization or SEO, creating quality content and modifying the keywords and key phrases used in your website content will greatly improve your searchability. For more information about writing quality web content, take a look at this other article.

3: Your Website Isn't Connected to Social Media

Did you know that over 1.5 billion people now use Facebook? Consider that there are only 7 billion people in the world. That's over 20% of the world population. In Canada and the United States alone there were 219 million active Facebook users per month in 2015. The point is this: if your website isn't integrated with your social media accounts, you're missing out on tons of potential site traffic. Modern content management systems (CMS) like Plone have social media integration built right in, helping to convert your social media visitors into website visitors, and vice versa.

A great feature that we are starting to see more and more is embedding social media feeds directly on your website. We like this tool because site visitors can see your social media posts without ever leaving your site. Any time visitors are rerouted off your site, there is a chance that many of them won't return. Keeping them around increases the chances that they will do what you want them to do, whether that means buying a product or simply reading an article.

2: It's Hard to Make Changes

If your website was custom made for you, you may find it difficult to make changes once the site has been launched-depending on the technology your designer used. Today, there are countless platforms, called content management systems (CMS), that designers utilize to create sites that are customizable to specific needs while also being easy to modify as needed. The templates within the Plone CMS for example make editing content pain-free, even if you're not as computer-savvy as Bill Gates. Check out this Acronym Monday article if you'd like to learn more about content management systems.

1: Your Website Isn't Mobile Friendly

There is no greater weakness for a website than being unresponsive for mobile devices. More than half of all web users today view content on their smartphones and tablets. If your site isn't optimized for users with those devices, you're losing tons of potential traffic through frustrated users (not to mention they're not likely to come back after their first bad experience). Most of us have been there: visiting the desktop version of a site on your phone and having to pinch around in order to find what you're looking for. Redoing your site with a responsive theme and a powerful content management system will ensure that your website works for everyone, broadening your reach.

Web design for a mobile-first world has become the standard. If you are having your website redesigned by a contractor, and they aren't offering you a mobile-friendly design as part of your project, they aren't doing their job right. Call them out on it, and ask for a mobile friendly design to be included.


All five examples don't have to be true for you to think about an update. If even one of these items is true of your website, you should consider giving your website a facelift, or you might consider a complete overhaul. Here at Wildcard Corp., we utilize state-of-the-art tools to provide you with a range of technology solutions to suit any need. Contact us now to get started on updating your website. If you have a problem, just name it. We'll solve it.

26 Jul 2016 8:04pm GMT

22 Jul 2016

feedPlanet Plone

Andreas Jung: Plone 5 and XML-Director 2.0 with Dropbox integration

After many months of pain with Plone 5.0, XML-Director 2.0 will be finally available for production soon. This screencast shows you how to integrated Plone via XML-Director with Dropbox (or other databases or (cloud) storages).

22 Jul 2016 6:39am GMT

18 Jul 2016

feedPlanet Plone

Martijn Faassen: Morepath 0.15 released!

Today the Morepath developers released Morepath 0.15 (CHANGES).

What is Morepath? Morepath is a Python web framework that is small, easy to learn, extensively documented, and insanely powerful.

This release is a smaller release without big visible changes, or big hood changes. Instead it polishes a lot of stuff. It also continues the trend with contributions from multiple core developers.

This release prepares the way for the next Morepath release. To this end we've deprecated a number of APIs. We are preparing a big change to the underlying Reg predicate dispatch library APIs that should make it less implicit, less magic, and make it perform slightly better. Stay tuned!

18 Jul 2016 12:16pm GMT

11 Jul 2016

feedPlanet Plone

Reinout van Rees: collective.recipe.sphinxbuilder buildout recipe works on python 3 now

I wanted to do a few small tweaks on collective.recipe.sphinxbuilder because it failed to install on python 3. I ended up as maintainer :-) It is now at https://github.com/reinout/collective.recipe.sphinxbuilder .

The only change needed was to tweak the way the readme was read in the setup.py and do a new release. Since then Thomas Khyn added windows support.

The documentation is now on readthedocs: http://collectiverecipesphinxbuilder.readthedocs.io/ . And the package is also tested on travis-ci.org.

11 Jul 2016 1:24pm GMT

04 Jul 2016

feedPlanet Plone

eGenix: Python Meeting Düsseldorf - 2016-07-06

The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.

Ankündigung

Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:

06.07.2016, 18:15 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf


Neuigkeiten

Bereits angemeldete Vorträge

Stefan Richthofer
"JyNI - Native CPython-Extensions in Jython"

Marc-Andre Lemburg
"Stand-Alone Applikationen mit eGenix PyRun"

Charlie Clark
"Eine kurze Einführung in SQLAlchemy: Was es ist und wie man es benutzen kann"

Jens Diemer
"PyLucid - ein Open Source CMS auf Django Basis"

Weitere Vorträge können gerne noch angemeldet werden. Bei Interesse, bitte unter info@pyddf.de melden.

Startzeit und Ort

Wir treffen uns um 18:15 Uhr im Bürgerhaus in den Düsseldorfer Arcaden.

Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.

Über dem Eingang steht ein großes "Schwimm' in Bilk" Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.

>>> Eingang in Google Street View

Einleitung

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

Programm

Das Python Meeting Düsseldorf nutzt eine Mischung aus Open Space und Lightning Talks, wobei die Gewitter bei uns auch schon mal 20 Minuten dauern können :-)

Lightning Talks können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit XGA Auflösung steht zur Verfügung.

Lightning Talk Anmeldung bitte formlos per EMail an info@pyddf.de

Kostenbeteiligung

Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.

Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.

Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.

Anmeldung

Da wir nur für ca. 20 Personen Sitzplätze haben, möchten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung.

Meeting Anmeldung bitte formlos per EMail an info@pyddf.de

Weitere Informationen

Weitere Informationen finden Sie auf der Webseite des Meetings:

http://pyddf.de/

Viel Spaß !

Marc-Andre Lemburg, eGenix.com

04 Jul 2016 9:00am GMT

30 Jun 2016

feedPlanet Plone

Reinout van Rees: Djangorecipe: easy test coverage reports

Code coverage reports help you see which parts of your code are still untested. Yes, it doesn't say anything about the quality of your tests, but at the least it tells you which parts of your code have absolute the worst kind of tests: those that are absent :-)

Ned Batchelder's coverage.py is the number one tool in python.

Note: I use buildout instead of pip to set up my projects. Reason: I can integrate extra automation that way. One of those extra automation steps is "djangorecipe": a recipe is a buildout plugin. It installs django, amongst others.

My previous setup

I use nose as a test runner. Easier test discovery was the reason at the time. Python's unittest2 is better at that, so it might be time to re-visit my choice. Especially as I just read that nose isn't really being maintained anymore.

A nice thing about nose is that you can have plugins. coverage.py is a standard plugin. And with django-nose you can easily use nose as a test runner instead of the standard django one. So once I put a few nose-related settings in my setup.cfg, coverage was run automatically every time I ran my tests. Including a quick coverage report.

The setup.cfg:

[nosetests]
cover-package = my_program
with-coverage = 1
cover-erase = 1
cover-html = 1
cover-html-dir = htmlcov

Running it:

$ bin/test
......................................
Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
djangorecipe                   0      0   100%
djangorecipe.binscripts       42     16    62%   25, 37-57
djangorecipe.boilerplate       1      0   100%
djangorecipe.recipe          115      0   100%
--------------------------------------------------------
TOTAL                        158     16    90%
----------------------------------------------------------------------
Ran 38 tests in 1.308s

OK

An important point here is that laziness is key. Just running bin/test (or an equivalent command) should run your tests and print out a quick coverage summary.

Note: bin/test is the normal test script if you set up a project with buildout, but it is effectively the same as python manage.py test.

New situation

"New" is relative here. Starting in django 1.7, you cannot use a custom test runner (like django-nose) anymore to automatically run your tests with coverage enabled. The new app initialization mechanism already loads your models.py, for instance, before the test runner gets called. So your models.py shows up as largely untested.

There's a bug report for this for django-nose.

There are basically two solutions. Either you run coverage separately:

$ coverage run python manage.py test
$ coverage report
$ coverage html_report

Ok. Right. You can guess what the prototypical programmer will do instead:

$ python manage.py test

That's easier. But you don't get coverage data.

The second alternative is to use the coverage API and modify your manage.py as shown in one of the answers. This is what I now build into buildout's djangorecipe (version 2.2.1).

bin/test now starts coverage recording before django gets called. It also prints out a report and export xml results (for recording test results in Jenkins, for instance) and html results. The only thing you need to do is to add coverage = true to your buildout config.

The details plus setup examples are in the 2.2.1 documentation.

(An advantage: I can now more easily move from nose to another test runner, if needed).

Not using buildout?

Most people use pip. Those using buildout for django will, I think, really appreciate this new functionality.

Using pip? Well, the basic idea still stands. You can use the call-coverage-API-in-manage.py approach just fine in your manage.py manually. Make it easy for yourself and your colleagues to get automatic coverage summaries!

30 Jun 2016 9:36am GMT