fidzu : framework

[Tox](https://tox.readthedocs.io) is a great tool for automated testing. We use it, not only to run matrix testing, but to run different types of tests in different environments, enabling us to parallelise our test runs, and get better reporting about what types of tests failed. Recently, we started using [Robot Framework](https://robotframework.org) for some automated UI testing. This needs to run a django server, and almost certainly wants to run against a different database. This will require our `tox -e robot` to drop the database if it exists, and then create it. Because we use [dj-database-url](https://pypi.org/project/dj-database-url/) to provide our database settings, our [Codeship](https://codeship.com/) configuration contains an environment variable set to `DATABASE_URL`. This contains the host, port and database name, as well as the username/password if applicable. However, we don't have the database name (or port) directly available in their own environment variables. Instead, I wanted to extract these out of the `postgres://user:password@host:port/dbname` string. My tox environment also needed to ensure that a distinct database was used for robot: {% highlight text %} [testenv:robot] setenv= CELERY_ALWAYS_EAGER=True DATABASE_URL={env:DATABASE_URL}_robot PORT=55002 BROWSER=headlesschrome whitelist_externals= /bin/sh commands= sh -c 'dropdb --if-exists $(echo {env:DATABASE_URL} | cut -d "/" -f 4)' sh -c 'createdb $(echo {env:DATABASE_URL} | cut -d "/" -f 4)' coverage run --parallel-mode --branch manage.py robot --runserver={env:PORT} {% endhighlight %} And this was working great. I'm also using the `$PG_USER` environment variable, which is supplied by Codeship, but that just clutters things up. However, when merged to our main repo, which has it's own codeship environment, tests were failing. It would complain about the database not being present when attempting to run the robot tests. It seems that we were using a different version of postgres, and thus were using a different port. So, how can we extract the port from the `$DATABASE_URL`? {% highlight text %} commands= sh -c 'dropdb --if-exists \ -p $(echo {env:DATABASE_URL} | cut -d "/" -f 3 | cut -d ":" -f 3) \ $(echo {env:DATABASE_URL} | cut -d "/" -f 4)' {% endhighlight %} Which is all well and good, until you have a `$DATABASE_URL` that omits the port... dropdb: error: missing required argument database name Ah, that would mean the command being executed was: $ dropdb --if-exists -p Eventually, I came up with the following: {% highlight text %} sh -c 'PG_PORT=$(echo {env:DATABASE_URL} | cut -d "/" -f 3 | cut -d ":" -f 3) \ dropdb --if-exists -p $\{PG_PORT:-5432} \ $(echo {env:DATABASE_URL} | cut -d "/" -f 4)' {% endhighlight %} Whew, that is a mouthful! We store the extracted value in a variable `PG_PORT`, and then use bash variable substitution (rather than tox variable substitution) to put it in, with a default value. But because of tox variable substitution, we need to escape the curly brace to allow it to be passed through to bash: `$\{PG_PORT:-5432}`. Also note that you'll need a space after this before a line continuation, because bash seems to strip leading spaces from the continued line.

02 Jun 2020 6:03pm GMT

One of my favorite job interview questions is this:

Write a function that returns tomorrow's date

This looks innocent enough for someone to suggest this as a solution:

import datetime

def tomorrow() -> datetime.date:
    return datetime.date.today() + datetime.timedelta(days=1)

This will work, but there is a followup question:

How would you test this function?

Before you move on.... take a second to think about your answer.

One of these pigeons is a mock
Photo by Pedro Figueras

Table of Contents

Naive Approach

The most naive approach to test a function that returns tomorrow's date is this:

# Bad
assert tomorrow() == datetime.date(2020, 4, 16)

This test will pass today, but it will fail on any other day.

Another way to test the function is this:

# Bad
assert tomorrow() == datetime.date.today() + datetime.timedelta(days=1)

This will also work, but there is an inherent problem with this approach. The same way you can't define a word in the dictionary using itself, you should not test a function by repeating its implementation.

Another problem with this approach is that it's only testing one scenario, for the day it is executed. What about getting the next day across a month or a year? What about the day after 2020-02-28?

The problem with both implementations is that today is set inside the function, and to simulate different test scenarios you need to control this value. One solution that comes to mind is to mock datetime.date, and try to set the value returned by today():

>>> from unittest import mock
>>> with mock.patch('datetime.date.today', return_value=datetime.date(2020, 1, 1)):
...     assert tomorrow() == datetime.date(2020, 1, 2)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/unittest/mock.py", line 1410, in __enter__
    setattr(self.target, self.attribute, new_attr)
TypeError: can't set attributes of built-in/extension type 'datetime.date'

As the exception suggests, built-in modules written in C cannot be mocked. The unittest.mock documentation specifically addresses this attempt to mock the datetime module. Apparently, this is a very common issue and the writers of the official documentation felt it's worth mentioning. They even go the extra mile and link to a blog post on this exact problem. The article is worth a read, and we are going to address the solution it presents later on.

Like every other problem in Python, there are libraries that provide a solution. Two libraries that stand out are freezegun and libfaketime. Both provide the ability to mock time at different levels. However, resorting to external libraries is a luxury only developers of legacy system can afford. For new projects, or projects that are small enough to change, there are other alternatives that can keep the project free of these dependencies.

Dependency Injection

The problem we were trying to solve with mock, can also be solved by changing the function's API:

import datetime

def tomorrow(asof: datetime.date) -> datetime.date:
    return asof + datetime.timedelta(days=1)

To control the reference time of the function, the time can be provided as an argument. This makes it easier to test the function in different scenarios:

import datetime
assert tomorrow(asof=datetime.date(2020, 5, 1))   == datetime.date(2020, 5, 2)
assert tomorrow(asof=datetime.date(2019, 12, 31)) == datetime.date(2020, 1, 1)
assert tomorrow(asof=datetime.date(2020, 2, 28))  == datetime.date(2020, 2, 29)
assert tomorrow(asof=datetime.date(2021, 2, 28))  == datetime.date(2021, 3, 1)

To remove the function's dependency on datetime.date.today, we provide today's date as an argument. This pattern of providing, or "injecting" dependencies into functions and objects is often called "dependency injection", or in short "DI".

Dependency Injection in The Wild

Dependency injection is a way to decouple modules from each other. As our previous example shows, the function tomorrow no longer depends on today.

Using dependency injection is very common and often very intuitive. It's very likely that you already use it without even knowing. For example, this article suggests that providing an open file to json.load is a form of dependency injection:

import json

with open('path/to/file.json', 'r') as f:
  data = json.load(f)

The popular test framework pytest builds its entire fixture infrastructure around the concept of dependency injection:

import pytest

@pytest.fixture
def one() -> int:
  return 1

@pytest.fixture
def two() -> int:
  return 2

def test_one_is_less_than_two(one: int, two: int) -> None:
  assert one < two

The functions one and two are declared as fixtures. When pytest executes the test function test_one_is_less_than_two, it will provide it with the values returned by the fixture functions matching the attribute names. In pytest, the injection is magically happening simply by using the name of a known fixture as an argument.

Dependency injection is not limited just to Python. The popular JavaScript framework Angular is also built around dependency injection:

@Component({
  selector: 'order-list',
  template: `...`
})
export class OrderListComponent {
  orders: Order[];

  constructor(orderService: OrderService) {
    this.orders = orderService.getOrders();
  }
}

Notice how the orderService is provided, or injected, to the constructor. The component is using the order service, but is not instantiating it.

Injecting Functions

Sometimes injecting a value is not enough. For example, what if we need to get the current date before and after some operation:

from typing import Tuple
import datetime

def go() -> Tuple[datetime.datetime, datetime.datetime]:
    started_at = datetime.datetime.now()
    # Do something ...
    ended_at = datetime.datetime.now()
    return started_at, ended_at

To test this function, we can provide the start time like we did before, but we can't provide the end time. One way to approach this is to make the calls to start and end outside the function. This is a valid solution, but for the sake of discussion we'll assume they need to be called inside.

Since we can't mock datetime.datetime itself, one way to make this function testable is to create a separate function that returns the current date:

from typing import Tuple
import datetime

def now() -> datetime.datetime:
  return datetime.datetime.now()

def go() -> Tuple[datetime.datetime, datetime.datetime]:
    started_at = now()
    # Do something ...
    ended_at = now()
    return started_at, ended_at

To control the values returned by the function now in tests, we can use a mock:

>>> from unittest import mock
>>> fake_start = datetime.datetime(2020, 1, 1, 15, 0, 0)
>>> fake_end = datetime.datetime(2020, 1, 1, 15, 1, 30)
>>> with mock('__main__.now', side_effect=[fake_start, fake_end]):
...    go()
(datetime.datetime(2020, 1, 1, 15, 0),
 datetime.datetime(2020, 1, 1, 15, 1, 30))

Another way to approach this without mocking, is to rewrite the function once again:

from typing import Callable, Tuple
import datetime

def go(
    now: Callable[[], datetime.datetime],
) -> Tuple[datetime.datetime, datetime.datetime]:
    started_at = now()
    # Do something ...
    ended_at = now()
    return started_at, ended_at

This time we provide the function with another function that returns a datetime. This is very similar to the first solution we suggested, when we injected the datetime itself to the function.

The function can now be used like this:

>>> go(datetime.datetime.now)
(datetime.datetime(2020, 4, 18, 14, 14, 5, 687471),
 datetime.datetime(2020, 4, 18, 14, 14, 5, 687475))

To test it, we provide a different function that returns known datetimes:

>>> fake_start = datetime.datetime(2020, 1, 1, 15, 0, 0)
>>> fake_end = datetime.datetime(2020, 1, 1, 15, 1, 30)
>>> gen = iter([fake_start, fake_end])
>>> go(lambda: next(gen))
(datetime.datetime(2020, 1, 1, 15, 0),
 datetime.datetime(2020, 1, 1, 15, 1, 30))

This pattern can be generalized even more using a utility object:

from typing import Iterator
import datetime

def ticker(
    start: datetime.datetime,
    interval: datetime.timedelta,
) -> Iterator[datetime.datetime]:
    """Generate an unending stream of datetimes in fixed intervals.

    Useful to test processes which require datetime for each step.
    """
    current = start
    while True:
        yield current
        current += interval

Using ticker, the test will now look like this:

>>> gen = ticker(datetime.datetime(2020, 1, 1, 15, 0, 0), datetime.timedelta(seconds=90))
>>> go(lambda: next(gen)))
(datetime.datetime(2020, 1, 1, 15, 0),
 datetime.datetime(2020, 1, 1, 15, 1, 30))

Fun fact: the name "ticker" was stolen from Go.

Injecting Values

The previous sections demonstrate injection of both values and functions. It's clear from the examples that injecting values is much simpler. This is why it's usually favorable to inject values rather than functions.

Another reason is consistency. Take this common pattern that is often used in Django models:

from django.db import models

class Order(models.Model):
    created = models.DateTimeField(auto_now_add=True)
    modified = models.DateTimeField(auto_now=True)

The model Order includes two datetime fields, created and modified. It uses Django's auto_now attribute to automatically set created when the object is saved for the first time, and auto_now_add to set modified every time the object is saved.

Say we create a new order and save it to the database:

>>> o = Order.objects.create()

Would you expect this test to fail:

>>> assert o.created == o.modified
False

This is very unexpected. How can an object that was just created have two different values for created and modified? Can you imagine what would happen if you rely on modified and created to be equal when an object was never changed, and actually use it to identify unchanged objects:

from django.db.models import F

# Wrong!
def get_unchanged_objects():
  return Order.objects.filter(created=F('modified'))

For the Order model above, this function will always return an empty queryset.

The reason for this unexpected behavior is that each individual DateTimeField is using django.timezone.now internally during save() to get the current time. The time between when the two fields are populated by Django causes the values to end up slightly different:

>>> o.created
datetime.datetime(2020, 4, 18, 11, 41, 35, 740909, tzinfo=<UTC>)

>>> o.modified
datetime.datetime(2020, 4, 18, 11, 41, 35, 741015, tzinfo=<UTC>)

If we treat timezone.now like an injected function, we understand the inconsistencies it may cause.

So, can this be avoided? Can created and modified be equal when the object is first created? I'm sure there are a lot of hacks, libraries and other such exotic solutions but the truth is much simpler. If you want to make sure these two fields are equal when the object is first created, you better avoid auto_now and auto_now_add:

from django.db import models

class Order(models.Model):
    created = models.DateTimeField()
    modified = models.DateTimeField()

Then, when you create a new instance, explicitly provide the values for both fields:

>>> from django.utils import timezone
>>> asof = timezone.now()
>>> o = Order.objects.create(created=asof, modified=asof)
>>> assert o.created == o.modified
>>> Order.objects.filter(created=F('modified'))
<QuerySet [<Order: Order object (2)>]>

To quote the "Zen of Python", explicit is better than implicit. Explicitly providing the values for the fields requires a bit more work, but this is a small price to pay for reliable and predictable data.

When to Instantiate Injected Values

Injecting values poses another interesting question, at what point should the value be set? The answer to this is "it depends", but there is a rule of thumb that is usually correct: values should be instantiated at the topmost level.

For example, if asof represents when an order is created, a website backend serving a store front may set this value when the request is received. In a normal Django setup, this means that the value should be set by the view. Another common example is a scheduled job. If you have jobs that use management commands, asof should be set by the management command.

Setting the values at the topmost level guarantees that the lower levels remain decoupled and easier to test. The level at which injected values are set, is the level that you will usually need to use mock to test. In the example above, setting asof in the view will make the models easier to test.

Other than testing and correctness, another benefit of setting values explicitly rather than implicitly, is that it gives you more control over your data. For example, in the website scenario, an order's creation date is set by the view immediately when the request is received. However, if you process a batch file from a large customer, the time in which the order was created may well be in the past, when the customer first created the files. By avoiding "auto-magically" generated dates, we can implement this by passing the past date as an argument.

Dependency Injection in Practice

The best way to understand the benefits of DI and the motivation for it is using a real life example.

IP Lookup

Say we want to try and guess where visitors to our Django site are coming from, and we decide to try an use the IP address from the request to do that. An initial implementation can look like this:

from typing import Optional
from django.http import HttpRequest
import requests

def get_country_from_request(request: HttpRequest) -> Optional[str]:
    ip = request.META.get('REMOTE_ADDR', request.META.get('HTTP_X_FORWARDED_FOR'))
    if ip is None or ip == '':
        return None

    response = requests.get(f'https://ip-api.com/json/{ip}')
    if not response.ok:
        return None

    data = response.json()
    if data['status'] != 'success':
        return None

    return data['countryCode']

This single function accepts an HttpRequest, tries to extract an IP address from the request headers, and then uses the requests library to call an external service to get the country code.

Let's try to use this function:

>>> from django.test import RequestFactory
>>> rf = RequestFactory()
>>> request = rf.get('/', REMOTE_ADDR='216.58.210.46')
>>> get_country_from_request(request)
'US'

OK, so it works. Notice that to use it we created an HttpRequest object using Django's RequestFactory

Let's try to write a test for a scenario when a country code is found:

import re
import json
import responses

from django.test import RequestFactory

rf = RequestFactory()

with responses.RequestsMock() as rsps:
    url_pattern = re.compile(r'http://ip-api.com/json/[0-9\.]+')
    rsps.add(responses.GET, url_pattern, status=200, content_type='application/json', body=json.dumps({
        'status': 'success',
        'countryCode': 'US'
    }))
    request = rf.get('/', REMOTE_ADDR='216.58.210.46')
    countryCode = get_country_from_request(request)
    assert countryCode == 'US'

The function is using the requests library internally to make a request to the external API. To mock the response, we used the responses library.

If you look at this test and feel like it's very complicated than you are right. To test the function we had to do the following:

• Generate a Django request using a RequestFactory.
• Mock a requests response using responses.
• Have knowledge of the inner works of the function (what url it uses).

That last point is where it gets hairy. To test the function we used our knowledge of how the function is implemented: what endpoint it uses, how the URL is structured, what method it uses and what the response looks like. This creates an implicit dependency between the test and the implementation. In other words, the implementation of the function cannot change without changing the test as well. This type of unhealthy dependency is both unexpected, and prevents us from treating the function as a "black box".

Also, notice that that we only tested one scenario. If you look at the coverage of this test you'll find that it's very low. So next, we try and simplify this function.

Assigning Responsibility

One of the techniques to make functions easier to test is to remove dependencies. Our IP function currently depends on Django's HttpRequest, the requests library and implicitly on the external service. Let's start by moving the part of the function that handles the external service to a separate function:

def get_country_from_ip(ip: str) -> Optional[str]:
    response = requests.get(f'http://ip-api.com/json/{ip}')
    if not response.ok:
        return None

    data = response.json()
    if data['status'] != 'success':
        return None

    return data['countryCode']

def get_country_from_request(request: HttpRequest) -> Optional[str]:
    ip = request.META.get('REMOTE_ADDR', request.META.get('HTTP_X_FORWARDED_FOR'))
    if ip is None or ip == '':
        return None

    return get_country_from_ip(ip)

We now have two functions:

• get_country_from_ip: receives an IP address and returns the country code.
• get_country_from_request: accepts a Django HttpRequest, extract the IP from the header, and then uses the first function to find the country code.

After splitting the function we can now search an IP directly, without crating a request:

>>> get_country_from_ip('216.58.210.46')
'US'
>>> from django.test import RequestFactory
>>> request = RequestFactory().get('/', REMOTE_ADDR='216.58.210.46')
>>> get_country_from_request(request)
'US'

Now, let's write a test for this function:

import re
import json
import responses

with responses.RequestsMock() as rsps:
    url_pattern = re.compile(r'http://ip-api.com/json/[0-9\.]+')
    rsps.add(responses.GET, url_pattern, status=200, content_type='application/json', body=json.dumps({
        'status': 'success',
        'countryCode': 'US'
    }))
    country_code = get_country_from_ip('216.58.210.46')
    assert country_code == 'US'

This test looks similar to the previous one, but we no longer need to use RequestFactory. Because we have a separate function that retrieves the country code for an IP directly, we don't need to "fake" a Django HttpRequest.

Having said that, we still want to make sure the top level function works, and that the IP is being extracted from the request correctly:

# BAD EXAMPLE!
import re
import json
import responses

from django.test import RequestFactory

rf = RequestFactory()
request_with_no_ip = rf.get('/')
country_code = get_country_from_request(request_with_no_ip)
assert country_code is None

We created a request with no IP and the function returned None. With this outcome, can we really say for sure that the function works as expected? Can we tell that the function returned None because it couldn't extract the IP from the request, or because the country lookup returned nothing?

Someone once told me that if to describe what a function does you need to use the words "and" or "or", you can probably benefit from splitting it. This is the layman's version of the Single-responsibility principle that dictates that every class or function should have just one reason to change.

The function get_country_from_request extracts the IP from a request and tries to find the country code for it. So, if the rule is correct, we need to split it up:

def get_ip_from_request(request: HttpRequest) -> Optional[str]:
    ip = request.META.get('REMOTE_ADDR', request.META.get('HTTP_X_FORWARDED_FOR'))
    if ip is None or ip == '':
        return None
    return ip


# Maintain backward compatibility
def get_country_from_request(request: HttpRequest) -> Optional[str]:
    ip = get_ip_from_request(request)
    if ip is None:
        return None
    return get_country_from_ip(ip)

To be able to test if we extract an IP from a request correctly, we yanked this part to a separate function. We can now test this function separately:

rf = RequestFactory()
assert get_ip_from_request(rf.get('/')) is None
assert get_ip_from_request(rf.get('/', REMOTE_ADDR='0.0.0.0')) == '0.0.0.0'
assert get_ip_from_request(rf.get('/', HTTP_X_FORWARDED_FOR='0.0.0.0')) == '0.0.0.0'
assert get_ip_from_request(rf.get('/', REMOTE_ADDR='0.0.0.0', HTTP_X_FORWARDED_FOR='1.1.1.1')) =='0.0.0.0'

With just these 5 lines of code we covered a lot more possible scenarios.

Using a Service

So far we've implemented unit tests for the function that extracts the IP from the request, and made it possible to do a country lookup using just an IP address. The tests for the top level function are still very messy. Because we use requests inside the function, we were forced to use responses as well to test it. There is nothing wrong with responses, but the less dependencies the better.

Invoking a request inside the function creates an implicit dependency between this function and the requests library. One way to eliminate this dependency is to extract the part making the request to a separate service:

import requests

class IpLookupService:

    def __init__(self, base_url: str) -> None:
        self.base_url = base_url

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        response = requests.get(f'{self.base_url}/json/{ip}')
        if not response.ok:
            return None

        data = response.json()
        if data['status'] != 'success':
            return None

        return data['countryCode']

The new IpLookupService is instantiated with the base url for the service, and provides a single function to get a country from an IP:

>>> ip_lookup_service = IpLookupService('http://ip-api.com')
>>> ip_lookup_service.get_country_from_ip('216.58.210.46')
'US'

Constructing services this way has many benefits:

• Encapsulate all the logic related to IP lookup
• Provides a single interface with type annotations
• Can be reused
• Can be tested separately
• Can be developed separately (as long as the API it provides remains unchanged)
• Can be adjusted for different environments (for example, use a different URL for test and production)

The top level function should also change. Instead of making requests on its own, it uses the service:

def get_country_from_request(
    request: HttpRequest,
    ip_lookup_service: IpLookupService,
) -> Optional[str]:
    ip = get_ip_from_request(request)
    if ip is None:
        return None
    return ip_lookup_service.get_country_from_ip(ip)

To use the function, we pass an instance of the service to it:

>>> ip_lookup_service = IpLookupService('http://ip-api.com')
>>> request = RequestFactory().get('/', REMOTE_ADDR='216.58.210.46')
>>> get_country_from_request(request, ip_lookup_service)
'US'

Now that we have full control of the service, we can test the top level function without using responses:

from unittest import mock
from django.test import RequestFactory

fake_ip_lookup_service = mock.create_autospec(IpLookupService)
fake_ip_lookup_service.get_country_from_ip.return_value = 'US'

request = RequestFactory().get('/', REMOTE_ADDR='216.58.210.46')

country_code = get_country_from_request(request, fake_ip_lookup_service)
assert country_code == 'US'

To test the function without actually making http requests we created a mock of the service. We then set the return value of get_country_from_ip, and passed the mock service to the function.

Changing Implementations

Another benefit of DI which is often mentioned, is the ability to completely change the underlying implementation of an injected service. For example, one day you discover that you don't have to use a remote service to lookup an IP. Instead, you can use a local IP database.

Because our IpLookupService does not leak its internal implementation, it's an easy switch:

from typing import Optional
import GeoIP

class LocalIpLookupService:
    def __init__(self, path_to_db_file: str) -> None:
        self.db = GeoIP.open(path_to_db_file, GeoIP.GEOIP_STANDARD)

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        return self.db.country_code_by_addr(ip)

The service API remained unchanged, so you can use it the same way as the old service:

>>> ip_lookup_service = LocalIpLookupService('/usr/share/GeoIP/GeoIP.dat')
>>> ip_lookup_service.get_country_from_ip('216.58.210.46')
'US'
>>> from django.test import RequestFactory
>>> request = RequestFactory().get('/', REMOTE_ADDR='216.58.210.46')
>>> get_country_from_request(request, ip_lookup_service)
'US'

The best part here is that the tests are unaffected. All the tests should pass without making any changes.

Typing Services

In the last section we cheated a bit. We injected the new service LocalIpLookupService into a function that expects an instance of IpLookupService. We made sure that these two are the same, but the type annotations are now wrong. We also used a mock to test the function which is also not of type IpLookupService. So, how can we use type annotations and still be able to inject different services?

from abc import ABCMeta
import GeoIP
import requests

class IpLookupService(metaclass=ABCMeta):
    def get_country_from_ip(self, ip: str) -> Optional[str]:
        raise NotImplementedError()


class RemoteIpLookupService(IpLookupService):
    def __init__(self, base_url: str) -> None:
        self.base_url = base_url

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        response = requests.get(f'{self.base_url}/json/{ip}')
        if not response.ok:
            return None

        data = response.json()
        if data['status'] != 'success':
            return None

        return data['countryCode']


class LocalIpLookupService(IpLookupService):
    def __init__(self, path_to_db_file: str) -> None:
        self.db = GeoIP.open(path_to_db_file, GeoIP.GEOIP_STANDARD)

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        return self.db.country_code_by_addr(ip)

We defined a base class called IpLookupService that acts as an interface. The base class defines the public API for users of IpLookupService. Using the base class, we can provide two implementations:

1. RemoteIpLookupService: uses the requests library to lookup the IP at an external.
2. LocalIpLookupService: uses the local GeoIP database.

Now, any function that needs an instance of IpLookupService can use this type, and the function will be able to accept any subclass of it.

Before we wrap things up, we still need to handle the tests. Previously we removed the test's dependency on responses, now we can ditch mock as well. Instead, we subclass IpLookupService with a simple implementation for testing:

from typing import Iterable

class FakeIpLookupService(IpLookupService):
    def __init__(self, results: Iterable[Optional[str]]):
        self.results = iter(results)

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        return next(self.results)

The FakeIpLookupService implements IpLookupService, and is producing results from a list of predefined results we provide to it:

from django.test import RequestFactory

fake_ip_lookup_service = FakeIpLookupService(results=['US'])
request = RequestFactory().get('/', REMOTE_ADDR='216.58.210.46')

country_code = get_country_from_request(request, fake_ip_lookup_service)
assert country_code == 'US'

The test no longer uses mock.

Using a Protocol

The form of class hierarchy demonstrated in the previous section is called "nominal subtyping". There is another way to utilize typing without classes, using Protocols:

from typing import Iterable, Optional
from typing_extensions import Protocol
import GeoIP
import requests


class IpLookupService(Protocol):
    def get_country_from_ip(self, ip: str) -> Optional[str]:
        pass


class RemoteIpLookupService:
    def __init__(self, base_url: str) -> None:
        self.base_url = base_url

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        response = requests.get(f'{self.base_url}/json/{ip}')
        if not response.ok:
            return None

        data = response.json()
        if data['status'] != 'success':
            return None

        return data['countryCode']


class LocalIpLookupService:
    def __init__(self, path_to_db_file: str) -> None:
        self.db = GeoIP.open(path_to_db_file, GeoIP.GEOIP_STANDARD)

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        return self.db.country_code_by_addr(ip)


class FakeIpLookupService:
    def __init__(self, results: Iterable[Optional[str]]):
        self.results = iter(results)

    def get_country_from_ip(self, ip: str) -> Optional[str]:
        yield from self.results

The switch from classes to protocols is mild. Instead of creating IpLookupService as a base class, we declare it a Protocol. A protocol is used to define an interface and cannot be instantiated. Instead, a protocol is used only for typing purposes. When a class implements the interface defined by the protocol, is means "Structural Subtyping" exits and the type check will validate.

In our case, we use a protocol to make sure an argument of type IpLookupService implements the functions we expect an IP service to provide.

So which to use? Some languages, like Java, use nominal typing exclusively, while other languages, like Go, use structural typing for interfaces. There are advantages and disadvantages to both ways, but we won't get into that here. In Python, nominal typing is easier to use and understand, so my recommendation is to stick to it, unless you need the flexibility afforded by protocols.

Nondeterminism and Side-Effects

If you ever had a test that one day just started to fail, unprovoked, or a test that fails once every blue moon for no apparent reason, it's possible your code is relying on something that is not deterministic. In the datetime.date.today example, the result of datetime.date.today relies on the current time which is always changing, hence it's not deterministic.

There are many sources of nondeterminism. Common examples include:

• Randomness
• Network access
• Filesystem access
• Database access
• Environment variables
• Mutable global variables

Dependency injection provides a good way to control nondeterminism in tests. The basic recipe is this:

1. Identify the source of nondeterminism and encapsulate it in a service: For example, TimeService, RandomnessService, HttpService, FilesystemService and DatabaseService.
2. Use dependency injection to access these services: Never bypass them by using datetime.now() and similar directly.
3. Provide deterministic implementations of these services in tests: Use a mock, or a custom implementation suited for tests instead.

If you follow the recipe diligently, your tests will not be affected by external circumstances and you will not have flaky tests!

Conclusion

Dependency injection is a design pattern just like any other. Developers can decide to what degree they want to take advantage of it. The main benefits of DI are:

• Decouple modules, functions and objects.
• Switch implementations, or support several different implementations.
• Eliminate nondeterminism from tests.

In the use-case above we took several twists and turns to illustrate a point, which might have caused the implementation to seem more complicated than it really is. In addition to that, searching for information about dependency injection in Python often result in libraries and packages than seem to completely change the way you structure your application. This can be very intimidating.

In reality, DI can be used sparingly and in appropriate places to achieve the benefits listed above. When implemented correctly, DI can make your code easier to maintain and to test.

01 Jun 2020 2:00am GMT

This week Symfony 5.1 was released. This major version contains hundreds of tweaks and improvements and tens of nice new features. Symfony also released its 3.4.41, 4.4.9 and 5.0.9 maintenance versions. In addition, this week we published the 700th issue of "A Week of Symfony". Thanks for reading us every week for more than 13 years, making it one of the longest-running blog series in the entire software industry.

Symfony development highlights

3.4 changelog:

• ce61bb0: [Security] fixed handling of CSRF logout error
• e1e314d: handle fetch mode deprecation of DBAL 2.11
• b7cd22c: [Console] added meaningful message when using ProcessHelper and Process is not installed
• a3c460f: [Form] updated Hungarian translations
• 8861ee1, 03cad5a: [Validator] updated Czech translation
• 2e8ae40: [Form] validated subforms in all validation groups
• ca66e6c: [PropertyAccess] fixed TypeError parsing again
• 8e40814: [Console] fixed QuestionHelper::disableStty()
• 84081ad: [Validator] never directly validate Existence (Required/Optional) constraints

4.4 changelog:

• b524e04: [Mime] allowed email message to have "To"
• 0d2bc7b: [ErrorHandler] fixed setting $trace to null in FatalError
• 96d2d19: [TwigBridge] fixed fallback html-to-txt body converter
• 5e0e9a8: [Validator] used Mime component to determine mime type for file validator
• bdb01db: [Security] fixed AbstractToken::hasUserChanged()

5.1 changelog:

• 3693875: [FrameworkBundle] fixed MicroKernelTrait for php 8
• 4739835: [HttpClient] fixed management of shorter-than-requested timeouts with AmpHttpClient
• 7f6934c: [Doctrine] fixed extra SQL support in Doctrine migrations
• 2ff26b7: [Messenger] changed the default notify timeout value for PostgreSQL
• af444f8: [Validator] use "allowedVariables" to configure the ExpressionLanguageSyntax constraint

Newest issues and pull requests

• [Messenger] Provide a safe single handler strategy
• [HttpClient] CachingHttpClient defeats concurrency
• [DependencyInjection] dump xml container definition without debug enabled
• Route controller linter

They talked about us

• Moving a Symfony app using Doctrine ORM from PostgreSQL to MySQL
• [Angular + Symfony] A quick website setup
• Symfony 5 confirmation email without FosUserBundle
• Securing your Symfony forms with Google reCAPTCHA v2
• The Bulletproof Event Naming For Symfony Event Dispatcher
• Configuring API Platform resources by following DDD principles
• Nuevo en Symfony 5.1: Nuevo sistema de seguridad

Call to Action

• Follow Symfony on Twitter and retweet this article.
• Subscribe to the Symfony blog RSS and never miss a Symfony story again.

31 May 2020 8:09am GMT

Symfony 5.1.0 has just been released. As for any other Symfony minor release, our backward compatibility promise applies and this means that you should be able to upgrade easily without changing anything in your code.

During the last couple of months, we've blogged about the great 5.1 new features. I highly recommend you to read the 47 articles about Symfony 5.1 as they contain the major changes for this new version:

• Added support for typed properties in PropertyInfo: In Symfony 5.1, the PropertyInfo component can extract information from the new typed properties introduced in PHP 7.4.
• Autowire public typed properties: In Symfony 5.1, public properties typed with classes related to services are autowired automatically, which is equivalent to the traditional setter injection.
• The "safe" HTTP Preference: Symfony 5.1 supports the "safe" HTTP preference defined in RFC 8674.
• Single command applications: In Symfony 5.1, creating single command console applications will be easier thanks to the new SingleCommandApplication class.
• Reusable sets of constraints: In Symfony 5.1 you can define reusable sets of constraints and you can also apply constraints sequentially to stop the validation as soon as one validation fails.
• Route annotations priority: In Symfony 5.1, routes defined as annotations can include a priority option to better control the URL matching without having to reorder the controller actions.
• Simpler security attributes: In Symfony 5.1, security attributes will be easier to use and a new IS_IMPERSONATOR attribute will simplify how to detect impersonating users.
• Improved UriSigner: In Symfony 5.1 we've improved the UriSigner service to make it autowireable and easier to use.
• Hostname Validator: Symfony 5.1 includes a new validator to check that a given value is valid as a hostname.
• Workflow improvements: In Symfony 5.1, the Workflow component has been improved to allow disable announce events, check if a workflow exists and explain blocked transitions.
• Simpler login in tests: Symfony 5.1 adds a new loginUser() method to simplify testing protected resources.
• Uid component: Symfony 5.1 will include a new component called Uid which generates UUIDs and ULIDs and provides some utilities for them.
• Translation improvements: Symfony 5.1 adds new features related to translations to configure the enabled locales, to improve the translation debug command, to support name attributes in Xliff2 and to allow translating each language into its own language.
• Form theme improvements: Symfony 5.1 adds a new form theme for Foundation 6, improves the translation of Bootstrap custom forms and allows to customize collection entries.
• Improved route configuration: In Symfony 5.1, configuring redirections and rendering templates from route configuration will be much simpler.
• Routing improvements: In Symfony 5.1, routes can be stateless, route conditions can include env vars and the request context is easier to configure.
• String improvements: In Symfony 5.1, the String component uses the Stringable interface from PHP 8, improves methods such as truncate() and adds useful methods such as containsAny() and reverse().
• Portable HTTP/2 implementation: In Symfony 5.1, the HttpClient component can use HTTP/2 without installing the cURL library thanks to the amp/http-client PHP package.
• Configurable PHP Preloading: In Symfony 5.1, you can configure PHP preloading by using some dependency injection tags to define which classes should be preloaded and which ones should not.
• Cursor control: Symfony 5.1 introduces new utilities to control the cursor in console commands.
• Simpler logout customization: In Symfony 5.1, logout customization will be much easier to implement thanks to a new LogoutEvent.
• Remote asset manifest: In Symfony 5.1, the asset manifest file can be stored in remote servers, which is ideal when using CDNs.
• Different hosts per locale: In Symfony 5.1, routes can define different hosts for each locale, which is useful to create fully internationalized URLs.
• Simpler service decoration: In Symfony 5.1, it's much easier to refer to the original service when using service decoration.
• Abstract service arguments: In Symfony 5.1, services can define abstract arguments whose values are calculated at runtime.
• Improved Microkernel: In Symfony 5.1 we've improved MicroKernelTrait to allow using the Kernel as a registry of autowired controllers and service factories.
• Stack decorators: Symfony 5.1 introduces a new syntax to stack decorators, providing a simpler alternative to classic middleware.
• New and Improved Integrations: Symfony 5.1 adds new integrations with third-party technologies and services in the Lock, Cache, Mailer, Messenger and Notifier components.
• URI Resolver: In Symfony 5.1, the DomCrawler component provides a new utility to resolve relative links into absolute URLs.
• Serializer improvements: Symfony 5.1 improves the serializer with a new @Ignore annotation, an unwrapping denormalizer, support for stdClass and scalar denormalization.
• Async AWS Support: Symfony 5.1 will use Async AWS, a modern alternative to the official Amazon AWS SDK for PHP, in components like Mailer and Messenger.
• Simpler Request Context: Symfony 5.1 allows to configure the entire request context using a single config parameter called "default_uri".
• ExpressionLanguage validator: In Symfony 5.1 you can validate the syntax of your ExpressionLanguage expressions without parsing or evaluating them.
• Server-side request forgery protection: Symfony 5.1 improves the HttpClient component to provide out-of-the-box protection against SSRF (Server-side request forgery).
• Deprecate public services into private services: In Symfony 5.1, you can turn public services into private services while keeping the backward compatibility of your code.
• Deprecated the Inflector component: In Symfony 5.1, the Inflector component has been deprecated and its logic has been moved into the String component.
• AtLeastOneOf validator: Symfony 5.1 adds a new AtLeastOneOf validator to check that a given value satisfies at least one of the given constraints.
• Access decision based on voter priority: Symfony 5.1 adds a new access decision strategy called "priority" which uses the decision from the highest priority security voter.
• OptionsResolver improvements: In Symfony 5.1, the OptionsResolver component has introduced an optional fluent interface to configure options and a way to provide extra debug information per option.
• Validator improvements: Symfony 5.1 validator adds support for three-letter country and language codes, adds an option to check that the collection items is divisible by a certain number and allows to define validation callables.
• Form improvements: Symfony 5.1 improves forms to validate HTML5 colors, allow rounding percent values, display HTML contents in labels, simplify timezone management, provide better defaults for inputmode option and other improvements in choice types.
• HttpFoundation improvements: Symfony 5.1 adds a new InputBag class, adds support for encrypting session data, adds more cache-control directives and allows to build cookie objects more easily.
• Automatic schema updates: Symfony 5.1 automates the creation of the database tables required by some components to store their contents (Messenger, Lock, Cache, etc.)
• Updated Security System: Symfony 5.1 includes a new Security system which simplifies the existing security features while giving developers more flexibility in a much more approachable way.
• Misc. improvements (Part 1): Some of the minor new features included in Symfony 5.1 will make your work easier, such as using constants for command exit codes, a better tempnam() function and customized random strings.
• Misc. improvements (Part 2): Symfony 5.1 will include a stand-alone YAML linter command, a better RoundRobin mailer transport, a separate log channel for deprecations and support for tags/metadata in mailers.
• Misc. improvements (Part 3): Symfony 5.1 will allow to include the severity in ConstraintViolationList, will add a new mailer log handler and a new deprecation contract, and will enable "dark mode" in exception pages.

31 May 2020 7:16am GMT

Symfony 5.1.0 has just been released. Here is a list of the most important changes:

• bug #37009 [Validator] use "allowedVariables" to configure the ExpressionLanguageSyntax constraint (@xabbuh)
• bug #37008 [Security] Fixed AbstractToken::hasUserChanged() (@wouterj)
• bug #36894 [Validator] never directly validate Existence (Required/Optional) constraints (@xabbuh)
• bug #37007 [Console] Fix QuestionHelper::disableStty() (@chalasr)
• bug #36865 [Form] validate subforms in all validation groups (@xabbuh)
• bug #36907 Fixes sprintf(): Too few arguments in form transformer (@pedrocasado)
• bug #36868 [Validator] Use Mime component to determine mime type for file validator (@pierredup)
• bug #37000 Add meaningful message when using ProcessHelper and Process is not installed (@l-vo)
• bug #36990 [Messenger] Change the default notify timeout value for PostgreSQL (@fabpot)
• bug #36995 [TwigBridge] fix fallback html-to-txt body converter (@nicolas-grekas)
• bug #36993 [ErrorHandler] fix setting $trace to null in FatalError (@nicolas-grekas)
• bug #36987 Handle fetch mode deprecation of DBAL 2.11. (@derrabus)
• bug #36984 [SecurityBundle] Fixed version constraint on security-core and security-guard (@wouterj)
• bug #36974 [Security] Fixed handling of CSRF logout error (@wouterj)

Want to upgrade to this new release? Because Symfony protects backwards-compatibility very closely, this should be quite easy. Use SymfonyInsight upgrade reports to detect the code you will need to change in your project and read our upgrade documentation to learn more.

Want to be notified whenever a new Symfony release is published? Or when a version is not maintained anymore? Or only when a security issue is fixed? Consider subscribing to the Symfony Roadmap Notifications.

31 May 2020 6:17am GMT

News

Pipenv new release

A major new release for Pipenv, the first since 2018!

pypa.io

PSF on Twitter: "The deadline to sign up to vote in @ThePSF upcoming election is May 31 AOE. To vote you need to be a PSF fellow, contributing, supporting and/or managing member. Membership info can be found here: https://t.co/W2H2ShVTDr"

The PSF's annual elections are this June. If you are not a member and want to be one, sign up before May 31 AOE to vote this year.

twitter.com

Articles

A tour of Django server setups

See what Django in production visually looks like with diagrams.

mattsegal.dev

Abusing the Django Admin app

Ken Whitesell walks us through automatically registering all of the models in our app while providing a custom ModelAdmin class to make their display useful. This technique is useful when dealing with a large number of models or a legacy database when you don't want to create 200 ModelAdmins.

dev.to

Handling SEO with Wagtail CMS

Adonis Simo walks us through how to make Wagtail more SEO friendly.

adonissimo.com

How to Get Hired as a Django Developer

Practical advice on finding work as a professional Python/Django programmer.

learndjango.com

Waiting in asyncio

While not directly written for Django, this cheat sheet explains the various options available in asyncio for waiting for results.

hynek.me

Debugging a Containerized Django App in VS Code

How to debug a containerized Django App in Visual Studio Code (VS Code).

testdriven.io

Sponsored Link

Two Scoops of Django 3.x: Best Practices for the Django Web Framework

The long-awaited update covers various tips, tricks, patterns, code snippets, and techniques of Django best practices.

feldroy.com

Podcasts

Django Chat - JWTs and AI - David Sanders

David is the author of the popular django-rest-framework-simplejwt package. They discuss authentication, JWTs, and his current role at an AI startup.

djangochat.com

Tutorials

Django Celery Tutorial Series

An approachable two-part tutorial series about setting up Django and Celery.

accordbox.com

Projects

phuoc-ng/csslayout

A very useful collection of popular layouts and patterns made with CSS. While not specifically Django related, HTML is unavoidable as a Django developer.

github.com

jonathan-s/django-sockpuppet

Build reactive applications with the django tooling you already know and love

github.com

Flimm/django-fullurl

Adds three template tags: fullurl, fullstatic, and buildfullurl.

github.com

django-fast-pagination

Faster Queries For Large Databases

github.com

django-grpc-framework

Django gRPC framework is a toolkit for building gRPC services, inspired by djangorestframework.

github.com

jrief/django-entangled

If you use Django's JSONField, django-entangled is the missing tool you need to build custom forms for these fields.

github.com

Shameless Plugs

LearnDjango.com - Free tutorials and premium books

learndjango.com

Work From Home Setups

Jeff created this moment featuring photos of our community's work-from-home desk setups in March as many of us were starting to transition to working from our homes. Take a photo and tag @webology if you want your setup added.

twitter.com

This RSS feed is published on https://django-news.com/. You can also subscribe via email.

29 May 2020 4:00pm GMT

What can you tell us about the team that built reportula.org?

The Team that made reportula.org is just one person. Pedro Oliveira, started Reportula when he needed a clean and fast web application that reported the Bacula Backups software of the company he works for. He has decided to open the project, and let it grow to full web application that is able to manage the Bacula Backups.

What can you tell us about the site in general? What are the goals of the site and the main audience?

Reportula is a php based web program that provides you a summarized output of jobs that have already run. It obtains its information from the Bacula's database. Aside from a nice graphical display, it provides summaries of your jobs, as well as graphs of job usage. This is a fairly high level bacula management tool.

The main goals were to create a web reporting tool for the bacula backups system, as I got further into the project it developed into something more than that. Right know it calculates average of bacula backups, it has time line history of backups. Imagine this scenario for example, if you use the crontab feature of reportula, you can see in time by how much data your backups infrastructure is growing.

Example. in 2011.05.01 if backups infrastructure stores 500 Tera bytes, in 2011.12.30 it stores 510 terabytes. This is very handy for us because with this feature you can predict the storage needs of your backups for the future.

What was your major consideration in using CodeIgniter for this?

I chose codeigniter because I needed an easy, fast, and supported PHP development framework. I found that with Codeigniter I could achieve that. This project was made in less than month.

Another nice thing about Codeigniter is that you don't have to "re-invent the wheel". Codeigniter has most of the thing that you need for an application already developed. All you have to do is connect the blocks which is very easy.

What is next on the plate for reportula.org? Any additional functionality you can tell us about?

On the plate for Reportula is user registrations, acls, and managing Bacula Backups like "bconsole".

Do you have any other information you'd like to share with the community? Tips from this project you'd like to share? Lessons you've learned?

First of all i think that Codeigniter is one of the best frameworks on the internet. I've tried them all (Cake, Yii, Symfony, Zend) they are all too complicated, too big, with lots of features and slow. They all had one problem BIG, STEEP LEARNING CURVE.

Codeigniter has less features than the others but you start making an application in less than 30 minutes. And what it does it does well! Even if you think you need a big framework after starting with codeigniter it cames to you that you don't need another framework to develop some applications. The lessons I learned are don't re-invent the wheel, Codeigniter does it and does it well, the community are nice, and always had support on the forum.

11 Nov 2011 10:19pm GMT

Hello Franck,

I'm sorry I have not yet encountered this problem before.

What version of eZ Pubish are you using?

What version of ezoe are you using?

If you have a set of reproducible instructions to recreate the issue using the latest community build available you may wish to consider filing an issue on http://issues.ez.no/ezoe

I hope this helps ...

Cheers,

Heath

10 Nov 2011 2:44am GMT

Edit your site.ini like that :

[Session]
SessionNameHandler=custom
SessionNamePerSiteAccess=disabled

Set a common cookie_domain for all your sub domains! To do that you can edit your apache virtual_host and add that line into it :

php_value session.cookie_domain ".mysite.com"

Thanks Yannick, it was really useful, but I needed to isolate groups of siteaccess under the same domain, it's possible using site.ini / [Session] / SessionNamePrefix. You just have to set the same value for each siteaccess you want to share the session.

Example :

I have siteaccesses siteA, siteB, siteC, siteD and siteE. I want siteA, site B and siteC to share a session, and siteD and siteE to share another session.

In

• settings/siteaccess/siteA/site.ini.append.php
• settings/siteaccess/siteB/site.ini.append.php
• settings/siteaccess/siteC/site.ini.append.php

[Session]
SessionNameHandler=custom
SessionNamePerSiteAccess=disabled
SessionNamePrefix=fooPrefix

In

• settings/siteaccess/siteD/site.ini.append.php
• settings/siteaccess/siteE/site.ini.append.php

[Session]
SessionNameHandler=custom
SessionNamePerSiteAccess=disabled
SessionNamePrefix=barPrefix

10 Nov 2011 2:14am GMT

Hello Amine,

I took a few minutes and wrote a custom owsimpleoperator template operator function on your behalf.
When you use this you should be able to do so without touching the code.

This solution provides you a new custom template operator named 'member_of_role' which you would use like this ... (note this is a simple static example you will want to use dynamic values instead of course.

{def $userID=10
        $roleID=1
        $isMemberOfRole=$userID|member_of_role( $roleID )}
 
{if $isMemberOfRole}
 
Success! This userID {$userID} is a member of this roleID {$roleID} ...
 
{else}
 
Failure! This userID {$userID} is *not* a member of this roleID {$roleID} ...
 
{/if}

The custom 'member_of_role' operator accepts UserID and RoleID (Both required) and fetches the user, user roles, iterates over user roles and compares RoleID param till it finds a match and returns true, false if no match is found / etc.

This operator uses builtin / core eZ Publish PHP class methods for the very best in terms of reliability and stability.

share.ez.no is terrible to use when trying to paste pre formated (whitespace, no tabs, plain text) source code within the ezoe custom plugin input which has been written outside of ezoe and pasted in.

This off and on is a real problem with this site. It's no wonder so few folks post source code within the forums these days ... So I have posted my example solution source code in a pastebin instead, http://ezpublish.pastebin.ca/2093464

Please let me know how this solution works for you.

One final note: This code can also be called statically within custom PHP code without making any changes.

Here is an example PHP call of the function which powers the operator: ''

Please remember to clear cache and regenerate autoloads be for using this solution within eZ Publish!

I hope this helps ...

Cheers,

Heath

Edit: I found some extra time later in the evening and tested the operator in templates and it worked perfectly (as well as via PHP directly). Best wishes!

10 Nov 2011 1:34am GMT

My MiniPlanet Zope product has been working steady and stable for some years, when suddenly a user request came along. Would it be possible to get a feed of all the items in a miniplanet? With this update it became possible. MiniPlanet is an old-styl...

09 Nov 2011 9:41am GMT

It has been quite some time that I announced_ that I'd be working as a freelancer. Lots of stuff had to be done in that time, but finally things are ready. I've founded my own little company and set up a small website: Welcome to Betabug Sirius!

07 Nov 2011 9:26am GMT

zope.testbrowser is a valuable tool for integration tests. Historically, the Zope community used to write quite a lot of doctests, but we at gocept have found them to be rather clumsy and too often yielding neither good tests nor good documentation. That's why we don't use doctest much anymore, and prefer plain unittest.TestCases instead. However, doctest has one very nice feature, ellipsis matching, that is really helpful for checking HTML output, since you can only make assertions about the parts that interest you. For example, given this kind of page:

>>> print browser.contents
<html>
  <head>
    <title>Simple Page</title>
  </head>
  <body>
    <h1>Simple Page</h1>
  </body>
</html>

If all you're interested in is that the <h1> is rendered properly, you can simply say:

>>> print browser.contents
<...<h1>Simple Page</h1>...

We've now ported this functionality to unittest, as assertEllipsis, in gocept.testing. Some examples:

self.assertEllipsis('...bar...', 'foo bar qux')
# -> nothing happens

self.assertEllipsis('foo', 'bar')
# -> AssertionError: Differences (ndiff with -expected +actual):
     - foo
     + bar

self.assertNotEllipsis('foo', 'foo')
# -> AssertionError: "Value unexpectedly matches expression 'foo'."

To use it, inherit from gocept.testing.assertion.Ellipsis in addition to unittest.TestCase.

03 Nov 2011 7:19am GMT

Every week we hear of really awesome places that CodeIgniter is being used. I want to start sharing those with the community-at-large. I will start by posting them here under a new Showcase Category with the hopes that any future revisions of CI.com will have a section for stuff like this. You guys and gals make some really cool stuff and deserve a platform to show it off.

So without further ado…

This showcase is an interview with Kyle Roseborrough about GoCart

What can you tell us about the GoCart team?

We have a pair of PHP developers who knew there was a better way to build a shipping cart. Noah (lead developer) has 6 years experience in PHP development and 4 years in CodeIgniter. Gabe has about 10 years experience in web application development. Kyle has been working in UI and management for 10 years.

What can we tell about the site in general?

GoCartdv.com was built to showcase GoCart and offer some basic information on the system.

What are the goals of the site and the main audience?

The main audience is CodeIgniter developers who are wanting a simple, scalable, CodeIgniter shopping cart. The goal is to get people involved in development to improve the cart and allow it to fully embody the goal of the project. To be easy to customize for developers and easy to use for end users/customers

What was your major consideration in using CodeIgniter for this?

CodeIgniter has great documentation and is easy to learn. We build lot of custom projects on CodeIgniter and it only made sense for us to build our cart on it. When looking for commerce solutions, we never found a suitable solution built on CodeIgniter so we decided to set out to do it on our own.

What is next on the plate for GoCart?

We really want GoCart to foster a great community of people contributing back to the roadmap and path the project will take. We want the focus to remain the same though "Easy to Customize, Easy to Use". It would be great if we could get enough people using.

Any additional functionality you can tell us about?

Well, not really. GoCart is intended to be a shopping cart, plain and simple. It does have some basic page and banner management and a whole slew of cart related features, but ultimately it's an ecommerce platform.

Do you have any other information you'd like to share with the community?

We built GoCart to be simple and scalable. As time goes on, we want the software to become easier and easier to use. We want GoCart to be scalable and to be able to work with new platforms as they come out. We feel that CodeIgniter and the CodeIgniter community is a huge benefit here. It enables developers to tie into a whole plethora of libraries, helpers and applications easily and support each other in the endeavor to make CodeIgniter better. Essentially, what's good for CodeIgniter is good for GoCart.

Tips from this project you'd like to share?

If you really want something, do it yourself. If it doesn't happen then you probably don't want it as bad as you think.

Lessons you've learned?

- Not every idea is a good one. Generally you need someone else around to discuss ideas and methods with. Collaboration is the best way to build a good application.
- No one knows what the next trend will be. Having a scalable platform that will adjust to a new set of tools and user demands is very important.

If you have a project that you would like to see in our showcase email me

02 Nov 2011 7:31pm GMT

Successful exit for German IT technology and now one of the services offered by eZ Systems.

With the acquisition of the German high-technology start-up, YOOCHOOSE, the Norwegian company eZ Systems AS is expanding its content management system, eZ Publish Enterprise, with one of the world's leading recommendation engines. YOOCHOOSE was founded in 2009 by Dr. Uwe Alkemper and Michael Friedmann, and convinced High-Tech Gruenderfonds to provide seed financing at an early stage. Behind YOOCHOOSE - a spin-off company from Deutsche Telekom Laboratories in Berlin - lies a high-performance, patented recommendation system, which enables companies to significantly increase their revenues on the Internet using personalized recommendations.

01 Nov 2011 1:07pm GMT

Through the acquisition of odoscope, eZ extends the powerful eZ Publish content management platform with behavior analysis optimizing the end-customer engagement.

26 Oct 2011 6:53am GMT

yum install nspr-devel
env LIBS="-lnss3 -lnspr4 -lplds4" ./configure
make && make install

export LD_PRELOAD="/usr/lib64/libnss3.so /usr/lib64/libnspr4.so /usr/lib64/libplds4.so"

23 Oct 2011 3:27pm GMT

eZ Content Cockpit is the latest extension from the eZ Market that offers a cohesive website overview to facilitate better editing and maintenance of dynamic content.

20 Oct 2011 7:43am GMT

So, a favor to ask of people: I'm working on a web application to help people manage their resumes. As I've gotten further in, I've realized I don't have an actual todo list for it. So, since I'm making this to be used by others, I'll ask everybody here:

What would you want to see? Currently, I've added enough code to allow the program to output something very close to my online resume ( http://www.icelus.org/ ). Next up, I have the following features on my todo list already:

• add docutils to output strings where appropriate (everywhere?)
• add login
• add ability to edit own resume
• add ability to compile new custom resumes
• add ability to output PDF
• add ability to output ODT
• add ability to output DOC
• add generic logo
• add generic photo / head (shadow) shot
• add ability to login using Google, Yahoo, Facebook, etc
• add ability to send resumes directly from Hiring Pond
• add ability to track where resumes have gone
• add ability to track job hunt progress
• add ability to display current employment status (employed, unemployed, contracting)
• add ability to display current employment seeking status (not at all, active, passive)
• add logo to qrcode
• change colors of qrcode
• change color scheme for all of hiring pond to sunset and water
• add fisherman silhouette as hiring pond logo

What else would you all want to see in order to make you want to use this?

18 Oct 2011 8:44pm GMT

The Zend Framework community is pleased to announce the immediate availability of Zend Framework 2.0.0beta1. Packages and installation instructions are available at: http://packages.zendframework.com/

18 Oct 2011 7:44am GMT

eZ Publish is a Web Content Management System that provides a platform to publish content via any channel. Its powerful presentation engine enables you to create websites and pages that display your content in a variety of renderings. Its powerful API directly and simply integrates your content with any web-enabled application on any device, such as the iPad, iPhone, or an Android device, without ever interfering with, or impacting the platform itself.

At the end of this tutorial, you will have learnt the basics of mobile application development for both iOS and Android platforms, consuming content from eZ Publish. CMS-side adjustments for the mobile channel will be acquired too. This cheatsheet will help you leverage the multichannel capabilities of eZ Publish, and its REST API in future projects, in a more systematic fashion.

13 Oct 2011 2:21pm GMT

We are happy to announce today that the user guide has had some significant improvements, and the first commit of these changes were just pushed today.

As many of you likely heard at CICON 2011, the Reactor team has had an internal project going on for some time to move the user guide to Sphinx. In addition to handling the tedium of generating page and document tables of contents, or maintaining internal links and references, the documentation is now easier to write, as you can simply focus on the content instead of markup and presentation. Don't forget syntax highlighting of PHP, HTML, CSS, and JavaScript in code samples. Based on ReStructured Text, it's also more human readable in a text editor than HTML is, which is likely where you spend most of your time. As an added benefit, Sphinx can output HTML, PDF, and even EPUB formats all from the same source files. We will likely be taking advantage of that at a later date.

But we didn't stop there, we also enlisted the thunderous powers of EllisLab's Chief Creative Officer, James Mathias for a style redesign. They are clean, easy to read, and beautiful.

Setting up your dev environment to work with Sphinx (if you want to render and output locally) is very easy, and takes about five minutes. For those that want to geek out, we have added a readme file to the user guide source folder so the step by step instructions are available right from GitHub.

Today marks the first commit with the new user guide to the unreleased develop branch, so you may encounter some bumps. Most notably are the code blocks, which pandoc lost our line breaks on, and some navigation issues as we experiment with different table of contents presentation and depth. We'll be cleaning these up prior to the next release (much is as simple as some line breaks and tabs), but feel free to pitch in and submit some pull requests if you see anything out of whack.

And lastly, for the first time ever, we have live nightly builds of documentation for the develop branch available at the CodeIgniter web site. Enjoy!

05 Oct 2011 7:23pm GMT

As recently some effort has been involved in improving the performances of TurboGears2, I was curious to see how much things improved. As usually, the test isn't really reliable in any way and was just for fun.

All the graphs report the request/sec the application has been able to perform on my computer with only 1 concurrent client. So higher is better.

Here is the comparison between TG2.0 and TG2dev (will be 2.1.4)

I also compared various setups with different template engines on TG2dev

The comparison happened on an application similar to the quickstarted one.
Actually as there is no database involved in this application the template engine impacts a lot and so was a good benchmark for the template engines themselves.

04 Oct 2011 3:35pm GMT

The Zend Framework team announces the immediate availability of Zend Framework's ALL ONES 1.11.11 release, the eleventh maintenance release in the 1.11 series. 1.11.11 includes around 30 bug fixes and may be downloaded from the Zend Framework site .

29 Sep 2011 7:52pm GMT

For those who haven't put the recurring event in their calendar, the Zend Framework Monthly Bug-hunt is here again! This Thursday, Friday and Saturday (the 22nd, 23rd and 24th of September), we'll be hosting our monthly bug hunt. For those of you unfamiliar with the event, each month, we organize the community to help reduce the number of open issues reported against the framework.

21 Sep 2011 3:07pm GMT

Beginners guide for learning how to use image maps in the ezwebin extension.

16 Aug 2011 12:40pm GMT

eZ Publish is a Web Content Management System that provides a platform to publish content via any channel. Its powerful presentation engine enables you to create websites and pages that display your content in a variety of renderings. Its powerful API directly and simply integrates your content with any web-enabled application on any device, such as the iPad, iPhone, or an Android device, without ever interfering with, or impacting the platform itself.

At the end of this tutorial, you will have learnt the basics of mobile application development for both iOS and Android platforms, consuming content from eZ Publish. CMS-side adjustments for the mobile channel will be acquired too. This cheatsheet will help you leverage the multichannel capabilities of eZ Publish, and its REST API in future projects, in a more systematic fashion.

07 Jul 2011 1:29pm GMT

One thing I often wished to have when using Git was the ability to autocomplete Git commands and branch names. As I had to learn this week from Markus Prinz' article A few of my Git tricks, tips and workflows, Git comes with an autocompletion script for the Bash shell. But to use the autocompletion, […]

06 Apr 2011 8:36am GMT

Till recently I always used a for-loop when I had to iterate over an array in JavaScript. For example: var myArray = [1, 2, 3, 4]; for (var i = 0; i < myArray.length; i++) { console.log(myArray[i]); } However, with ECMAScript 5 the Array object itself got some methods for iteration purposes. With those methods […]

01 Apr 2011 2:51pm GMT

From emails I receive it seems like there is a bit of confusion about what the terms 2-legged OAuth and 3-legged OAuth mean. I hope I can clear up this confusion with this article (and don't contribute more to the confusion…). In short, they describe two different usage scenarios of OAuth involving two respectively three […]

10 Jan 2011 5:30pm GMT

We can use immediately an experienced Cake developer for assisting us with developing a multi-lingual application that needs some Jake/Joomla (css) integration. We have continuously Cake projects and prefer to work with a team of individual developers in multiple time zones. Please show me that you are experienced, affordable and have at least 24 hours available per week (40 is better ;-).

04 Mar 2010 11:54am GMT