fidzu : Python

By Vasudev Ram

XMLtoPDFBook is a publishing tool I created, that allows you to create simple PDF ebooks from text content in XML files.

I had blogged about XMLtoPDFBook earlier, here:

Create PDF books with XMLtoPDFBook

and here:

XMLtoPDFBook now supports chapter numbers and names

Today I added some support for a Table of Contents feature to XMLtoPDFBook. Here is the updated program:

# XMLtoPDFBook2.py

# A program to convert a book in XML text format to a PDF book.
# Uses xtopdf and ReportLab.

# Author: Vasudev Ram - http://www.dancingbison.com
# Version: v0.2

#--------------------------------------------------------------------

# imports

import sys
import os
import string
import time

from PDFWriter import PDFWriter

try:
    import xml.etree.cElementTree as ET
except ImportError:
    import xml.etree.ElementTree as ET

#--------------------------------------------------------------------

# global variables

sysargv = None

#--------------------------------------------------------------------

def debug(message):
    sys.stderr.write(message + "\n")

#--------------------------------------------------------------------

def get_xml_filename(sysargv):
    return sysargv[1]

#--------------------------------------------------------------------

def get_pdf_filename(sysargv):
    return sysargv[2]

#--------------------------------------------------------------------

def XMLtoPDFBook():

    debug("Entered XMLtoPDFBook()")

    global sysargv

    # Get command-line arguments.
    xml_filename = get_xml_filename(sysargv)
    debug("xml_filename: " + xml_filename)
    pdf_filename = get_pdf_filename(sysargv)
    debug("pdf_filename: " + pdf_filename)

    # Parse the XML file.
    try:
        tree = ET.ElementTree(file=xml_filename)
        debug("tree = " + repr(tree))
    except Exception:
        sys.stderr.write("Error: caught exception in ET.ElementTree(file)")
        sys.exit(1)

    # Get the tree root.
    root = tree.getroot()
    debug("root.tag = " + root.tag)
    if root.tag != "book":
        debug("Error: Root tag is not 'book'")
        sys.exit(1)

    # Initialize the table of contents list.
    toc = []
    # Initialize the chapters list.
    chapters = []

    # Traverse the tree, extracting needed data into variables.
    debug("-" * 60)
    for root_child in root:
        if root_child.tag != "chapter":
            debug("Error: root_child tag is not 'chapter'")
            sys.exit(1)
        chapter = root_child
        #debug(chapter.text)
        chapters.append(chapter.text)
        try:
            chapter_name = chapter.attrib['name']
        except KeyError:
            chapter_name = ""
        toc.append(chapter_name)
        debug("-" * 60)

    # Create and set some fields of a PDFWriter.
    pw = PDFWriter(pdf_filename)
    pw.setFont("Courier", 12)
    pw.setFooter("Generated by XMLtoPDFBook. Copyright 2013 Vasudev Ram")

    # Write the TOC.
    pw.setHeader("Table of Contents")
    chapter_num = 0
    debug("Chapter names")
    for chapter_name in toc:
        debug(chapter_name)
        chapter_num += 1
        pw.writeLine(str(chapter_num) + ": " + chapter_name)
    pw.savePage()

    # Write the chapters.
    chapter_num = 0
    for chapter in chapters:
        chapter_num += 1
        pw.setHeader("Chapter " + str(chapter_num) + ": " + toc[chapter_num - 1])
        lines = chapter.split("\n")
        for line in lines:
            pw.writeLine(line)
        pw.savePage()

    pw.close()

    debug("Exiting XMLtoPDFBook()")

def main():

    debug("Entered main()")

    global sysargv
    sysargv = sys.argv

    # Check for right number of arguments.
    if len(sysargv) != 3:
        sys.exit(1)

    XMLtoPDFBook()

    debug("Exiting main()")

#--------------------------------------------------------------------

if __name__ == "__main__":
    try:
        main()
    except Exception, e:
        sys.stderr.write("Error: caught Exception" + str(e))
        sys.exit(1)

#--------------------------------------------------------------------

You can run it as follows:

python XMLtoPDFBook2.py vi_quickstart2.xml vi_quickstart2.pdf 

where I've used my vi quickstart tutorial, first written for Linux For You magazine, as the input XML file.

Here is a screenshot of the first page of the resulting PDF ebook - the Table of Contents:

And here is a screenshot Chapter 3 of the book:

I've pushed the code (as file XMLtoPDFBook2.py) to my xtopdf project on Bitbucket.

Enjoy.

- Vasudev Ram - Dancing Bison Enterprises

Contact Page

05 Dec 2013 5:30am GMT

By Vasudev Ram

XMLtoPDFBook is a publishing tool I created, that allows you to create simple PDF ebooks from text content in XML files.

I had blogged about XMLtoPDFBook earlier, here:

Create PDF books with XMLtoPDFBook

and here:

XMLtoPDFBook now supports chapter numbers and names

Today I added some support for a Table of Contents feature to XMLtoPDFBook. Here is the updated program:

# XMLtoPDFBook2.py

# A program to convert a book in XML text format to a PDF book.
# Uses xtopdf and ReportLab.

# Author: Vasudev Ram - http://www.dancingbison.com
# Version: v0.2

#--------------------------------------------------------------------

# imports

import sys
import os
import string
import time

from PDFWriter import PDFWriter

try:
    import xml.etree.cElementTree as ET
except ImportError:
    import xml.etree.ElementTree as ET

#--------------------------------------------------------------------

# global variables

sysargv = None

#--------------------------------------------------------------------

def debug(message):
    sys.stderr.write(message + "\n")

#--------------------------------------------------------------------

def get_xml_filename(sysargv):
    return sysargv[1]

#--------------------------------------------------------------------

def get_pdf_filename(sysargv):
    return sysargv[2]

#--------------------------------------------------------------------

def XMLtoPDFBook():

    debug("Entered XMLtoPDFBook()")

    global sysargv

    # Get command-line arguments.
    xml_filename = get_xml_filename(sysargv)
    debug("xml_filename: " + xml_filename)
    pdf_filename = get_pdf_filename(sysargv)
    debug("pdf_filename: " + pdf_filename)

    # Parse the XML file.
    try:
        tree = ET.ElementTree(file=xml_filename)
        debug("tree = " + repr(tree))
    except Exception:
        sys.stderr.write("Error: caught exception in ET.ElementTree(file)")
        sys.exit(1)

    # Get the tree root.
    root = tree.getroot()
    debug("root.tag = " + root.tag)
    if root.tag != "book":
        debug("Error: Root tag is not 'book'")
        sys.exit(1)

    # Initialize the table of contents list.
    toc = []
    # Initialize the chapters list.
    chapters = []

    # Traverse the tree, extracting needed data into variables.
    debug("-" * 60)
    for root_child in root:
        if root_child.tag != "chapter":
            debug("Error: root_child tag is not 'chapter'")
            sys.exit(1)
        chapter = root_child
        #debug(chapter.text)
        chapters.append(chapter.text)
        try:
            chapter_name = chapter.attrib['name']
        except KeyError:
            chapter_name = ""
        toc.append(chapter_name)
        debug("-" * 60)

    # Create and set some fields of a PDFWriter.
    pw = PDFWriter(pdf_filename)
    pw.setFont("Courier", 12)
    pw.setFooter("Generated by XMLtoPDFBook. Copyright 2013 Vasudev Ram")

    # Write the TOC.
    pw.setHeader("Table of Contents")
    chapter_num = 0
    debug("Chapter names")
    for chapter_name in toc:
        debug(chapter_name)
        chapter_num += 1
        pw.writeLine(str(chapter_num) + ": " + chapter_name)
    pw.savePage()

    # Write the chapters.
    chapter_num = 0
    for chapter in chapters:
        chapter_num += 1
        pw.setHeader("Chapter " + str(chapter_num) + ": " + toc[chapter_num - 1])
        lines = chapter.split("\n")
        for line in lines:
            pw.writeLine(line)
        pw.savePage()

    pw.close()

    debug("Exiting XMLtoPDFBook()")

def main():

    debug("Entered main()")

    global sysargv
    sysargv = sys.argv

    # Check for right number of arguments.
    if len(sysargv) != 3:
        sys.exit(1)

    XMLtoPDFBook()

    debug("Exiting main()")

#--------------------------------------------------------------------

if __name__ == "__main__":
    try:
        main()
    except Exception, e:
        sys.stderr.write("Error: caught Exception" + str(e))
        sys.exit(1)

#--------------------------------------------------------------------

You can run it as follows:

python XMLtoPDFBook2.py vi_quickstart2.xml vi_quickstart2.pdf 

where I've used my vi quickstart tutorial, first written for Linux For You magazine, as the input XML file.

Here is a screenshot of the first page of the resulting PDF ebook - the Table of Contents:

And here is a screenshot Chapter 3 of the book:

I've pushed the code (as file XMLtoPDFBook2.py) to my xtopdf project on Bitbucket.

Enjoy.

- Vasudev Ram - Dancing Bison Enterprises

Contact Page

05 Dec 2013 5:30am GMT

05 Dec 2013 1:45am GMT

05 Dec 2013 1:45am GMT

It seems a number of people are interested in what my development setup looks like. I'm constantly emailed questions asking what IDE I use, what OS, what packages, etc. To stem the tide, I'll outline my dev setup here.

Editor: Vim

It should come as no surprise that I use a pure text editor (rather than an IDE) for writing code. Outside of Java, it seems like most professional developers use either vim or emacs. The reason, I assume, is that they're always available (especially in vi's case). No matter what company I work for, regardless of the platform they're on, I know I'll always be able to use Vim.

In addition, a decade with Vim makes me a very fast editor of code. I don't care how quick you are with your favorite IDE and a mouse; I'm faster. Every time I try to move to an IDE, I switch back after a few days if it doesn't have vi key bindings. So many Vim commands have become muscle memory that it doesn't feel like I'm using an editor to write code. Vim feels like an extension of myself, and I shape the code at will.

Here's a picture of what my Vim window looks like (on OS X):

That being said, there are a couple of rather nice Vim packages that I use.

In no particular order, here are the packages I use:

• vundle Essential as a package manager (I can finally list all my Vim dependencies in my .vimrc file). Vim packages are installable directly from GitHub, which is a nice touch.
• fugitive Best git interface for Vim
• vim-repeat Use . to repeat much more than simple inserts or deletes
• ctrlp.vim A buffer/file/mru/tag explorer with fuzzy text matching
• vim-markdown Markdown syntax highlighting for Vim
• gundo.vim Visualize and traverse your undo tree. A must
• YouCompleteMe
• syntastic These two together make the absolute best autocomplete package around
• vim-colors-solarized Solarized color scheme for Vim
• powerline Powerline integration for Vim

OS: Arch Linux

My OS of choice is Linux. Specifically, Arch Linux. Having used Linux on the desktop for the past ten years, Arch is exactly what I'm looking for in a distro: don't force choices on me, stay at the bleeding edge, and get out of my way. While some bemoan insurmountable issues they encountered during simple upgrades, I can count on one hand the number of problems I've run into that took more than 15 minutes to solve. Arch also acts as a good way to really learn Linux.

But Also OS X

Since I run Linux on most of my personal PCs, I never had the need for a Mac. When I joined AppNexus, I was given a Mac Book Pro Retina. I was lost until I found Terminal (and later iTerm2). OS X, especially in comparison to Windows, is great. I do, however, hate the Command key and the reliance on Homebrew/Macports for all my favorite software.

Shell: zsh

I was a bash user by default, as that's what I most commonly found installed by default at the various companies I worked for. Once I discovered zsh, however, there was no going back. First with "oh-my-zsh" and now with "prezto", I have an amazing shell setup. The tab-completion alone is worth the price of admission (here's what I see when I type sh <Tab>:

zsh just does everything right. I also use powerline in both the shell and vim. It's suitably awesome (looks great, nice git integration, etc) without slowing things down.

If you're interested in my configuration files (.vimrc, .zshrc, etc), they're publicly available on GitHub under my config_files repo.

Font: Adobe's Source Code Pro

When this font came out, everyone went nuts for it. Well, I'm still nuts for it. In my opinion, Source Code Pro is the single best programming font, hands-down. It's almost too pretty.

Python Version: Mercurial Latest

One of the first things I do when I get a new machine is hg clone the cPython repo and build both a 2.7.x and 3.x version of the interpreter. I like to stay at the bleeding edge (frequently pulling down changes) and not rely on whatever happens to be installed on the system (I'm looking at you CentOS 5). One of my former employers had very strict rules about what software could be installed on servers (including dev servers), so I got very good at wget-ing the source, ./configure-ing it and make install-ing it (and dealing with all the problems that popped up as a result). Therefore, I'm quite happy to build the interpreter from scratch and have done so a dozen times.

Python Tools: A Bunch

Here are the Python packages and tools I can't live without:

Requests

No surpise here...

iPython

The best interpreter experience around

virtualenvwrapper

Makes working with virtualenvs a breeze

BeautifulSoup

HTML/XML manipulation library

Flask

The most user-friendly web framework in the Python ecosystem

pip

Duh...

SQLAlchemy

Is there even a competitor for database ORMs? I honestly don't know the answer to that.

tox, mock, py.test, coverage, pylint, pep8

Makes testing actually enjoyable

Pandas / numpy

Awesome library for data analysis

selfspy

Really cool "Quantified Self" daemon

pdb

For someone coming from C/C++, ipdb is a godsend. Simply the best Python debugger

Cython

Sometimes you just gotta write C

pypy

It's fun to write your own language in RPython and get a JIT-enabled interpreter for it for free

HTTPie

A better curl than curl

Wrap-Up

In the end, I have a pretty boring setup: zsh and vim on Linux. The key thing to realize is just how powerful each of those tools are on their own. Combined, they make for an excellent development experience.

04 Dec 2013 7:20pm GMT

It seems a number of people are interested in what my development setup looks like. I'm constantly emailed questions asking what IDE I use, what OS, what packages, etc. To stem the tide, I'll outline my dev setup here.

Editor: Vim

It should come as no surprise that I use a pure text editor (rather than an IDE) for writing code. Outside of Java, it seems like most professional developers use either vim or emacs. The reason, I assume, is that they're always available (especially in vi's case). No matter what company I work for, regardless of the platform they're on, I know I'll always be able to use Vim.

In addition, a decade with Vim makes me a very fast editor of code. I don't care how quick you are with your favorite IDE and a mouse; I'm faster. Every time I try to move to an IDE, I switch back after a few days if it doesn't have vi key bindings. So many Vim commands have become muscle memory that it doesn't feel like I'm using an editor to write code. Vim feels like an extension of myself, and I shape the code at will.

Here's a picture of what my Vim window looks like (on OS X):

That being said, there are a couple of rather nice Vim packages that I use.

In no particular order, here are the packages I use:

• vundle Essential as a package manager (I can finally list all my Vim dependencies in my .vimrc file). Vim packages are installable directly from GitHub, which is a nice touch.
• fugitive Best git interface for Vim
• vim-repeat Use . to repeat much more than simple inserts or deletes
• ctrlp.vim A buffer/file/mru/tag explorer with fuzzy text matching
• vim-markdown Markdown syntax highlighting for Vim
• gundo.vim Visualize and traverse your undo tree. A must
• YouCompleteMe
• syntastic These two together make the absolute best autocomplete package around
• vim-colors-solarized Solarized color scheme for Vim
• powerline Powerline integration for Vim

OS: Arch Linux

My OS of choice is Linux. Specifically, Arch Linux. Having used Linux on the desktop for the past ten years, Arch is exactly what I'm looking for in a distro: don't force choices on me, stay at the bleeding edge, and get out of my way. While some bemoan insurmountable issues they encountered during simple upgrades, I can count on one hand the number of problems I've run into that took more than 15 minutes to solve. Arch also acts as a good way to really learn Linux.

But Also OS X

Since I run Linux on most of my personal PCs, I never had the need for a Mac. When I joined AppNexus, I was given a Mac Book Pro Retina. I was lost until I found Terminal (and later iTerm2). OS X, especially in comparison to Windows, is great. I do, however, hate the Command key and the reliance on Homebrew/Macports for all my favorite software.

Shell: zsh

I was a bash user by default, as that's what I most commonly found installed by default at the various companies I worked for. Once I discovered zsh, however, there was no going back. First with "oh-my-zsh" and now with "prezto", I have an amazing shell setup. The tab-completion alone is worth the price of admission (here's what I see when I type sh <Tab>:

zsh just does everything right. I also use powerline in both the shell and vim. It's suitably awesome (looks great, nice git integration, etc) without slowing things down.

If you're interested in my configuration files (.vimrc, .zshrc, etc), they're publicly available on GitHub under my config_files repo.

Font: Adobe's Source Code Pro

When this font came out, everyone went nuts for it. Well, I'm still nuts for it. In my opinion, Source Code Pro is the single best programming font, hands-down. It's almost too pretty.

Python Version: Mercurial Latest

One of the first things I do when I get a new machine is hg clone the cPython repo and build both a 2.7.x and 3.x version of the interpreter. I like to stay at the bleeding edge (frequently pulling down changes) and not rely on whatever happens to be installed on the system (I'm looking at you CentOS 5). One of my former employers had very strict rules about what software could be installed on servers (including dev servers), so I got very good at wget-ing the source, ./configure-ing it and make install-ing it (and dealing with all the problems that popped up as a result). Therefore, I'm quite happy to build the interpreter from scratch and have done so a dozen times.

Python Tools: A Bunch

Here are the Python packages and tools I can't live without:

Requests

No surpise here...

iPython

The best interpreter experience around

virtualenvwrapper

Makes working with virtualenvs a breeze

BeautifulSoup

HTML/XML manipulation library

Flask

The most user-friendly web framework in the Python ecosystem

pip

Duh...

SQLAlchemy

Is there even a competitor for database ORMs? I honestly don't know the answer to that.

tox, mock, py.test, coverage, pylint, pep8

Makes testing actually enjoyable

Pandas / numpy

Awesome library for data analysis

selfspy

Really cool "Quantified Self" daemon

pdb

For someone coming from C/C++, ipdb is a godsend. Simply the best Python debugger

Cython

Sometimes you just gotta write C

pypy

It's fun to write your own language in RPython and get a JIT-enabled interpreter for it for free

HTTPie

A better curl than curl

Wrap-Up

In the end, I have a pretty boring setup: zsh and vim on Linux. The key thing to realize is just how powerful each of those tools are on their own. Combined, they make for an excellent development experience.

04 Dec 2013 7:20pm GMT

A feature that has recently landed in git master on GitHub and will be released in 1.2 is the ability to log into Zato's web-admin using OpenID.

This lets one make use of an already existing Single Sign-On (SSO) infrastructure instead of requiring Zato admins to memorize additional credentials.

Here's how to enable it:

• Open the config file at /path/to/web/admin/config/repo/web-admin.conf

• Change the OPENID_SSO_SERVER_URL to a URL your SSO server uses

• Stop and start the web admin

• For each user in web admin:

  • Make sure the user has been already created, let's say it's 'myuser'
  • Issue the new zato update openid command, for instance

  % zato update openid /path/to/web/admin myuser https://sso.example.com/myuser
  OK
  %
  

Where

• zato update openid - the command to invoke
• /path/to/web/admin - path to web admin's top-level directory
• myuser - username whose OpenID claimed ID should be set
• https://sso.example.com/myuser - claimed ID of the user

No restarts are needed after updating a given user's credentials.

Note that enabling SSO disables regular password based authentication. To revert to the latter, set OPENID_SSO_SERVER_URL to "" and restart web admin.

04 Dec 2013 4:21pm GMT

A feature that has recently landed in git master on GitHub and will be released in 1.2 is the ability to log into Zato's web-admin using OpenID.

This lets one make use of an already existing Single Sign-On (SSO) infrastructure instead of requiring Zato admins to memorize additional credentials.

Here's how to enable it:

• Open the config file at /path/to/web/admin/config/repo/web-admin.conf

• Change the OPENID_SSO_SERVER_URL to a URL your SSO server uses

• Stop and start the web admin

• For each user in web admin:

  • Make sure the user has been already created, let's say it's 'myuser'
  • Issue the new zato update openid command, for instance

  % zato update openid /path/to/web/admin myuser https://sso.example.com/myuser
  OK
  %
  

Where

• zato update openid - the command to invoke
• /path/to/web/admin - path to web admin's top-level directory
• myuser - username whose OpenID claimed ID should be set
• https://sso.example.com/myuser - claimed ID of the user

No restarts are needed after updating a given user's credentials.

Note that enabling SSO disables regular password based authentication. To revert to the latter, set OPENID_SSO_SERVER_URL to "" and restart web admin.

04 Dec 2013 4:21pm GMT

The log messages generated by a piece software tell a story: what, where, when, even why and how if you're lucky. The readers of this story are more often than not other programs: monitoring systems, performance tools, or just filtering the messages down to something a human can actually comprehend. Unfortunately the output of most logging systems is ill-suited to being read by programs. Even worse, most logging systems omit critical information that both humans and their programs need. As part of my work at HybridCluster I've been working on fixing that.

Problem #1: Text is hard to search

Let's say you want to find all the log messages about a specific person. A first pass of log messages might look like this:

Sir James Chettam was going to dine at the Grange to-day with another gentleman whom the girls had never seen, and about whom Dorothea felt some venerating expectation.
…
If Miss Brooke ever attained perfect meekness, it would not be for lack of inward fire.

You could do a text search for log messages containing the text "Dorothea", but this is likely to fail for some types of searches. You might want to searching for actions involving dinner, but then you would need to search for "dine" and "dinner" and perhaps other words well. A library like structlog that can generate structured log messages will solve this first problem. You could define a "person" field in your messages and then you can search for all messages where person == "Dorothea" as well as other structured queries.

Problem #2: Referring to Entities

Every time a log message is written out you need to decide how to refer to the objects being logged. In the messages we saw above "Dorothea" and "Miss Brooke" are in fact different identifiers for the same person. Having structured messages doesn't help us find all messages about a specific entity if the object is referred to inconsistently. What you need is infrastructure for converting specific kinds of objects in your code to fields in your structured log messages. Then you can just say "log a message that refers to this Person" and that reusable code will make sure the correct identifier is generated.

Problem #3: Actions

Most log messages in your program are going to involve actions:

Not long after that dinner-party she had become Mrs. Casaubon, and was on her way to Rome.

A marriage has a beginning and eventually an end. The end may be successful, presuming "until death do us part" is a form of success, or a failure. The same is true of all actions, much like function calls in Python are started and eventually return a result or throw an exception. Actions may of course span multiple function calls or extended periods of time.

Actions also generate other actions: a marriage leads to a trip to Rome, the trip to Rome might lead to a visit to the Vatican Museum, and so on. Other unrelated actions are occurring at the same time, resulting in a forest of actions, with root actions that grow a tree of child actions.

You might want to trace an action from beginning to end, e.g. to measure how long it took to run. You might want to know what high-level action caused a particular unexpected low-level action. You might want to know what actions a specific entity was involved with. None of these are possible in most logging systems since they have no concept of actions to begin with.

Those are the problems. In my following posts I will talk about the solutions, being built as part of a Python library called Eliot. We hope to open source Eliot at some point in the future, but the general design ideas should be applicable anywhere.

The post Logging as Storytelling appeared first on HybridCluster.

04 Dec 2013 3:08pm GMT

The log messages generated by a piece software tell a story: what, where, when, even why and how if you're lucky. The readers of this story are more often than not other programs: monitoring systems, performance tools, or just filtering the messages down to something a human can actually comprehend. Unfortunately the output of most logging systems is ill-suited to being read by programs. Even worse, most logging systems omit critical information that both humans and their programs need. As part of my work at HybridCluster I've been working on fixing that.

Problem #1: Text is hard to search

Let's say you want to find all the log messages about a specific person. A first pass of log messages might look like this:

Sir James Chettam was going to dine at the Grange to-day with another gentleman whom the girls had never seen, and about whom Dorothea felt some venerating expectation.
…
If Miss Brooke ever attained perfect meekness, it would not be for lack of inward fire.

You could do a text search for log messages containing the text "Dorothea", but this is likely to fail for some types of searches. You might want to searching for actions involving dinner, but then you would need to search for "dine" and "dinner" and perhaps other words well. A library like structlog that can generate structured log messages will solve this first problem. You could define a "person" field in your messages and then you can search for all messages where person == "Dorothea" as well as other structured queries.

Problem #2: Referring to Entities

Every time a log message is written out you need to decide how to refer to the objects being logged. In the messages we saw above "Dorothea" and "Miss Brooke" are in fact different identifiers for the same person. Having structured messages doesn't help us find all messages about a specific entity if the object is referred to inconsistently. What you need is infrastructure for converting specific kinds of objects in your code to fields in your structured log messages. Then you can just say "log a message that refers to this Person" and that reusable code will make sure the correct identifier is generated.

Problem #3: Actions

Most log messages in your program are going to involve actions:

Not long after that dinner-party she had become Mrs. Casaubon, and was on her way to Rome.

A marriage has a beginning and eventually an end. The end may be successful, presuming "until death do us part" is a form of success, or a failure. The same is true of all actions, much like function calls in Python are started and eventually return a result or throw an exception. Actions may of course span multiple function calls or extended periods of time.

Actions also generate other actions: a marriage leads to a trip to Rome, the trip to Rome might lead to a visit to the Vatican Museum, and so on. Other unrelated actions are occurring at the same time, resulting in a forest of actions, with root actions that grow a tree of child actions.

You might want to trace an action from beginning to end, e.g. to measure how long it took to run. You might want to know what high-level action caused a particular unexpected low-level action. You might want to know what actions a specific entity was involved with. None of these are possible in most logging systems since they have no concept of actions to begin with.

Those are the problems. In my following posts I will talk about the solutions, being built as part of a Python library called Eliot. We hope to open source Eliot at some point in the future, but the general design ideas should be applicable anywhere.

The post Logging as Storytelling appeared first on HybridCluster.

04 Dec 2013 3:08pm GMT

As of today, PythonAnywhere supports Python 3 web apps for all frameworks that work with Python 3. We'd love to hear any feedback people have.

To create a Python 3 web app, just click the "Add a new web app" button. If you have an existing web app that you'd like to switch over from Python 2.7 to Python 3, drop us a line with the "Send feedback" option.

04 Dec 2013 1:21pm GMT

As of today, PythonAnywhere supports Python 3 web apps for all frameworks that work with Python 3. We'd love to hear any feedback people have.

To create a Python 3 web app, just click the "Add a new web app" button. If you have an existing web app that you'd like to switch over from Python 2.7 to Python 3, drop us a line with the "Send feedback" option.

04 Dec 2013 1:21pm GMT

This post demonstrates how you can configure a very decent environment for Python development with Emacs, in just 60 seconds.

A dream is now true: The first time you start Emacs, it automagically downloads and configures all plugins you need. Emacs is then just ready to work and you can start typing code immediately.

For the impatient

1. Save configuration files you eventually have!

$ cd $HOME
$ tar cpf dot-emacs.ORIGINAL.tar.gz .emacs .emacs.d
$ mv .emacs dot-emacs.ORIGINAL
$ mv .emacs.d dot-emacs.d.ORIGINAL

2. Remove any Emacs configuration files you eventually have.

$ rm -r -f .emacs .emacs.d

3. Install Python libraries

This should be done preferably inside a virtual environment.

$ workon py276 #-- py276 is a virtualenv I'm using
$ pip install epc
$ pip install jedi
$ pip install elpy

4. Download my .emacs file onto your home folder.

$ cd $HOME
$ wget https://raw.github.com/frgomes/dot-emacs/master/dot-emacs.el
$ ln -s dot-emacs.el .emacs

4. Start emacs. It will configure itself when it first run!

$ emacs test.py

Features in a nutshell

• python-mode, cython-mode and nxml-mode: ditto
• jedi: provides auto completion
• flymake: highlight syntax errors as you type

Contribute

Please let me know if you find issues. In particular, I don't have Windoze boxes, so the automagic configuration thing was never tested on it.

This script was designed to run on Emacs 23 onwards but only tested on Emacs 24. Please let me know if you find issues.

Please point out typos and bad English.

You can also suggest plugins or tools I missed. This is very much appreciated and may benefit my workflow as well :)
So... thanks a lot for your suggestion!

Known Issues

If you are behind firewall, you may (or may not) face download problems which involves HTTPS protocol. As far as I know, this is a bug on a third party library which Emacs depends on.

If Emacs opens the message window and vomits hundreds of errors coming from file cython-mode.el ... that's because your proxy server refused the https request and returned an error message in HTML. It's easy to fix this issue:

$ cd $HOME/.emacs.d/plugins
$ rm cython-mode.el
$ wget https://raw.github.com/cython/cython/master/Tools/cython-mode.el

Chances are that now cython-mode.el is OK, since wget performs the request the way it needs to be in order to work properly.


That's it: ready for coding in 60 seconds :)

Cheers

-- Richard Gomes

04 Dec 2013 8:47am GMT

This post demonstrates how you can configure a very decent environment for Python development with Emacs, in just 60 seconds.

A dream is now true: The first time you start Emacs, it automagically downloads and configures all plugins you need. Emacs is then just ready to work and you can start typing code immediately.

For the impatient

1. Save configuration files you eventually have!

$ cd $HOME
$ tar cpf dot-emacs.ORIGINAL.tar.gz .emacs .emacs.d
$ mv .emacs dot-emacs.ORIGINAL
$ mv .emacs.d dot-emacs.d.ORIGINAL

2. Remove any Emacs configuration files you eventually have.

$ rm -r -f .emacs .emacs.d

3. Install Python libraries

This should be done preferably inside a virtual environment.

$ workon py276 #-- py276 is a virtualenv I'm using
$ pip install epc
$ pip install jedi
$ pip install elpy

4. Download my .emacs file onto your home folder.

$ cd $HOME
$ wget https://raw.github.com/frgomes/dot-emacs/master/dot-emacs.el
$ ln -s dot-emacs.el .emacs

4. Start emacs. It will configure itself when it first run!

$ emacs test.py

Features in a nutshell

• python-mode, cython-mode and nxml-mode: ditto
• jedi: provides auto completion
• flymake: highlight syntax errors as you type

Contribute

Please let me know if you find issues. In particular, I don't have Windoze boxes, so the automagic configuration thing was never tested on it.

This script was designed to run on Emacs 23 onwards but only tested on Emacs 24. Please let me know if you find issues.

Please point out typos and bad English.

You can also suggest plugins or tools I missed. This is very much appreciated and may benefit my workflow as well :)
So... thanks a lot for your suggestion!

Known Issues

If you are behind firewall, you may (or may not) face download problems which involves HTTPS protocol. As far as I know, this is a bug on a third party library which Emacs depends on.

If Emacs opens the message window and vomits hundreds of errors coming from file cython-mode.el ... that's because your proxy server refused the https request and returned an error message in HTML. It's easy to fix this issue:

$ cd $HOME/.emacs.d/plugins
$ rm cython-mode.el
$ wget https://raw.github.com/cython/cython/master/Tools/cython-mode.el

Chances are that now cython-mode.el is OK, since wget performs the request the way it needs to be in order to work properly.


That's it: ready for coding in 60 seconds :)

Cheers

-- Richard Gomes

04 Dec 2013 8:47am GMT

Do you remember this? Well, now I have a similar but quite different case: I want to restore the files' times. Why? Read me out.

Since a long time I'm doing backups to an external drive. For that I simply use rsync. When I bought my second laptop, I took the chance to test restoring from that backup, so I just did a last backup in the old machine and a restore in the new one. This allowed me to add the last few things that were missing. After a few more weeks finding missing things, doing more backup/restore cycles between the old and new laptops, I was happy.

But there was something that I didn't realize till a few months later. The file times were not backed up, and clearly not restored. This didn't affect me in any way, at least until I tried to a new post to this glob.

See, this glob is generated with ikiwiki. ikiwiki uses the files' mtime to generate the posts' times, and uses that to sort them and to assign them as belonging to some year or month. With the mtimes lost, ikiwiki was assigning all the older posts to some day in Oct, 2012, which was the date of the last backup/restore cycle.

Frustrated (yes, sometimes I have a low threshold), I left it like that: no planet was flooded because in any case the rss feeds didn't really change, and nobody would notice and complain about it. In fact, nobody did.[1] Have in mid that as I kept my old laptop as a server, and that I didn't clean up all the files I keep in my backup, I still had the original mtimes in the original files.

Yesterday, playing around with unrelated things, I came across the post I mentioned at the beginning of this one, and had the urge to really fix that mess. Today I started looking at it, and found that this command was some part of it:

find src/projects/glob/work/ -name *.mdwn | xargs stat --format '%X %Y %Z %n'

but processing its output in bash seemed like a burden. Luckily, I have the solution for that, a project that I started for exactly that same reason: bash sucks at manipulating data.

So I grabbed my editor, punched keys for some 7 minutes, run 3 or 4 tests, making sure that everything would go smooth, run once more on the live system, rebuilt my glob[2] and voilà! It's fixed now.

How long is the code? Not much really:

from os import utime

with remote ('192.168.0.42', allow_agent=False, password='XXXXXXXX') as fds:
    # find al files (not only posts), get its times and path
    with cd ('src/projects/glob/work'):
        find ('.', '-name', '*.mdwn') | xargs ('stat', format='%X %Y %Z %n')

(i, o, e)= fds

for line in e.readlines ():
    print (line.decode ('utf-8'), end='')

for line in o.readlines ():
    data= line.split ()
    access, modif, change= [ int (x) for x in data[:3] ]
    file_path= data[3].decode ('utf-8')
    if _e (file_path):
        utime (file_path, times= (access, modif))
    else:
        print ("%s not found!" % file_path)

Notice that this example is even more complex than initially thought: it ssh's into the server to fetch the original times.

This makes one thing clear: I have to improve a lot the remote() API. For instance, so far I have no way to know if the remote commands executed fine or not, and the way to read the lines is still ugly and not homogeneous with how other ayrton functions/executables work.

Also note that this does not restore the ctimes, as I hadn't found a way to do it, but didn't really did a lot of effort. I don't really need them.

Of course, I also fixed my backup and restore scripts :)

[1] I was this >< close of adding #foreveralone to that sentence. I think I'll resist for a couple more years.

[2] Don't forget, ikiwiki compiles static sites.

ayrton python sysadmin

03 Dec 2013 10:19pm GMT

Do you remember this? Well, now I have a similar but quite different case: I want to restore the files' times. Why? Read me out.

Since a long time I'm doing backups to an external drive. For that I simply use rsync. When I bought my second laptop, I took the chance to test restoring from that backup, so I just did a last backup in the old machine and a restore in the new one. This allowed me to add the last few things that were missing. After a few more weeks finding missing things, doing more backup/restore cycles between the old and new laptops, I was happy.

But there was something that I didn't realize till a few months later. The file times were not backed up, and clearly not restored. This didn't affect me in any way, at least until I tried to a new post to this glob.

See, this glob is generated with ikiwiki. ikiwiki uses the files' mtime to generate the posts' times, and uses that to sort them and to assign them as belonging to some year or month. With the mtimes lost, ikiwiki was assigning all the older posts to some day in Oct, 2012, which was the date of the last backup/restore cycle.

Frustrated (yes, sometimes I have a low threshold), I left it like that: no planet was flooded because in any case the rss feeds didn't really change, and nobody would notice and complain about it. In fact, nobody did.[1] Have in mid that as I kept my old laptop as a server, and that I didn't clean up all the files I keep in my backup, I still had the original mtimes in the original files.

Yesterday, playing around with unrelated things, I came across the post I mentioned at the beginning of this one, and had the urge to really fix that mess. Today I started looking at it, and found that this command was some part of it:

find src/projects/glob/work/ -name *.mdwn | xargs stat --format '%X %Y %Z %n'

but processing its output in bash seemed like a burden. Luckily, I have the solution for that, a project that I started for exactly that same reason: bash sucks at manipulating data.

So I grabbed my editor, punched keys for some 7 minutes, run 3 or 4 tests, making sure that everything would go smooth, run once more on the live system, rebuilt my glob[2] and voilà! It's fixed now.

How long is the code? Not much really:

from os import utime

with remote ('192.168.0.42', allow_agent=False, password='XXXXXXXX') as fds:
    # find al files (not only posts), get its times and path
    with cd ('src/projects/glob/work'):
        find ('.', '-name', '*.mdwn') | xargs ('stat', format='%X %Y %Z %n')

(i, o, e)= fds

for line in e.readlines ():
    print (line.decode ('utf-8'), end='')

for line in o.readlines ():
    data= line.split ()
    access, modif, change= [ int (x) for x in data[:3] ]
    file_path= data[3].decode ('utf-8')
    if _e (file_path):
        utime (file_path, times= (access, modif))
    else:
        print ("%s not found!" % file_path)

Notice that this example is even more complex than initially thought: it ssh's into the server to fetch the original times.

This makes one thing clear: I have to improve a lot the remote() API. For instance, so far I have no way to know if the remote commands executed fine or not, and the way to read the lines is still ugly and not homogeneous with how other ayrton functions/executables work.

Also note that this does not restore the ctimes, as I hadn't found a way to do it, but didn't really did a lot of effort. I don't really need them.

Of course, I also fixed my backup and restore scripts :)

[1] I was this >< close of adding #foreveralone to that sentence. I think I'll resist for a couple more years.

[2] Don't forget, ikiwiki compiles static sites.

ayrton python sysadmin

03 Dec 2013 10:19pm GMT

I'm working on a new Python web microframework called Morepath, as I've mentioned before. Here's the code and here's a draft quickstart. Morepath is a microframework with a difference: it's small and easy to learn like the others, but has special super powers under the hood.

One of those super powers is Reg, which along with Morepath's model/view separation makes it easy to write reusable views. But in this article I'll talk about another super power: Morepath's application reuse patterns.

We'll talk about how Morepath lets you isolate applications, extend and override applications, and compose applications together. Morepath tries to make these possibilities simple, even obvious. Morepath strives to make the possible as obvious as possible.

app = morepath.App()

@app.model(model=User, path='user/{username}',
           variables=lambda user: { 'username': user.username })
def get_user(username):
    return query_for_user(username)

@app.view(model=User)
def render_user(request, model):
    return "User: %s" % model.username

other_app = morepath.App()
@other_app.model(model=User, path='different_path/{username}')
def get_user(username):
    return different_query_for_user(username)

@other_app.view(model=User)
def render_user(request, model):
    return "Differently Displayed User: %s" % model.username

@app.view(model=User, name='edit')
def edit_user(request, model):
    return 'Edit user: %s' % model.username

extended_app = morepath.App(extends=[app])

@extended_app.view(model=User, name='edit')
def edit_user(request, model):
    return 'Edit user: %s' % model.username

@extended_app.view(model=User)
def render_user_differently(request, model):
    return 'Different view for user: %s' % model.username

@extended_app.model(model=OtherUser, path='user/{username}')
def get_user_differently(username):
    return OtherUser(username)

@app.model(model=Wiki, path='user/{username}/wiki')
def get_wiki(username):
    return wiki_for_user(username)

@app.view(model=Wiki)
def wiki_default_view(request, model):
    return "Default view for wiki"

wiki_app = morepath.App()

@wiki_app.model(model=Wiki, path='{wiki_id}')
def get_wiki(wiki_id):
    return query_wiki(wiki_id)

@wiki_app.view(model=Wiki)
def wiki_default_view(request, model):
    return "Default view for wiki"

@app.mount(app=wiki_app, path='user/{username}/wiki')
def mount_wiki(username):
    return {
       'wiki_id': get_wiki_id_for_username(username)
    }

@wiki_app.model(model=Wiki, path='')
def get_wiki(wiki_id):
    return query_wiki(wiki_id)

wiki_app.context(wiki_id=5)

03 Dec 2013 3:10pm GMT

I'm working on a new Python web microframework called Morepath, as I've mentioned before. Here's the code and here's a draft quickstart. Morepath is a microframework with a difference: it's small and easy to learn like the others, but has special super powers under the hood.

One of those super powers is Reg, which along with Morepath's model/view separation makes it easy to write reusable views. But in this article I'll talk about another super power: Morepath's application reuse patterns.

We'll talk about how Morepath lets you isolate applications, extend and override applications, and compose applications together. Morepath tries to make these possibilities simple, even obvious. Morepath strives to make the possible as obvious as possible.

app = morepath.App()

@app.model(model=User, path='user/{username}',
           variables=lambda user: { 'username': user.username })
def get_user(username):
    return query_for_user(username)

@app.view(model=User)
def render_user(request, model):
    return "User: %s" % model.username

other_app = morepath.App()
@other_app.model(model=User, path='different_path/{username}')
def get_user(username):
    return different_query_for_user(username)

@other_app.view(model=User)
def render_user(request, model):
    return "Differently Displayed User: %s" % model.username

@app.view(model=User, name='edit')
def edit_user(request, model):
    return 'Edit user: %s' % model.username

extended_app = morepath.App(extends=[app])

@extended_app.view(model=User, name='edit')
def edit_user(request, model):
    return 'Edit user: %s' % model.username

@extended_app.view(model=User)
def render_user_differently(request, model):
    return 'Different view for user: %s' % model.username

@extended_app.model(model=OtherUser, path='user/{username}')
def get_user_differently(username):
    return OtherUser(username)

@app.model(model=Wiki, path='user/{username}/wiki')
def get_wiki(username):
    return wiki_for_user(username)

@app.view(model=Wiki)
def wiki_default_view(request, model):
    return "Default view for wiki"

wiki_app = morepath.App()

@wiki_app.model(model=Wiki, path='{wiki_id}')
def get_wiki(wiki_id):
    return query_wiki(wiki_id)

@wiki_app.view(model=Wiki)
def wiki_default_view(request, model):
    return "Default view for wiki"

@app.mount(app=wiki_app, path='user/{username}/wiki')
def mount_wiki(username):
    return {
       'wiki_id': get_wiki_id_for_username(username)
    }

@wiki_app.model(model=Wiki, path='')
def get_wiki(wiki_id):
    return query_wiki(wiki_id)

wiki_app.context(wiki_id=5)

03 Dec 2013 3:10pm GMT

Our young coders event happens on Saturday, February 22, 2014, and is limited to 24 spots. Registration is available at our Eventbrite page at 12 noon today (12/3). Our young coder event is lead by the uber talented Katie Cunningham!

Having lead young coders tutorials at both PyCon 2013, PyOhio, and others, we are beyond thrilled to welcome her to our little event. Learn more about our young coders event at our website. Registration for young coders will be free, and tickets will be available today at noon. Please don't register unless you are 100% able to attend!

Katie Cunningham is a Python developer at Cox Media Group. She's a fervent advocate for Python, Open Source Software, and teaching more people how to program. She's a frequent speaker at open source conferences, such as PyCon and DjangoCon, speaking on beginners topics such as someone's first site in the cloud, and making a site that is accessible to everyone.

She also helps organize PyLadies in the DC area, a program designed to increase diversity in the Python community. She has taught classes for the organization, bringing novices from installation to writing their first app in 48 hours.

Katie is an active blogger at her website ( http://therealkatie.net), covering issues such as Python, accessibility, and the trials and tribulations of working from home.

Katie is also the author of Python in 24 Hours and the Accessibility Handbook.

We are still looking for a sponsor for young coders, so if you are interested you can learn more at our website.

03 Dec 2013 1:29pm GMT

Our young coders event happens on Saturday, February 22, 2014, and is limited to 24 spots. Registration is available at our Eventbrite page at 12 noon today (12/3). Our young coder event is lead by the uber talented Katie Cunningham!

Having lead young coders tutorials at both PyCon 2013, PyOhio, and others, we are beyond thrilled to welcome her to our little event. Learn more about our young coders event at our website. Registration for young coders will be free, and tickets will be available today at noon. Please don't register unless you are 100% able to attend!

Katie Cunningham is a Python developer at Cox Media Group. She's a fervent advocate for Python, Open Source Software, and teaching more people how to program. She's a frequent speaker at open source conferences, such as PyCon and DjangoCon, speaking on beginners topics such as someone's first site in the cloud, and making a site that is accessible to everyone.

She also helps organize PyLadies in the DC area, a program designed to increase diversity in the Python community. She has taught classes for the organization, bringing novices from installation to writing their first app in 48 hours.

Katie is an active blogger at her website ( http://therealkatie.net), covering issues such as Python, accessibility, and the trials and tribulations of working from home.

Katie is also the author of Python in 24 Hours and the Accessibility Handbook.

We are still looking for a sponsor for young coders, so if you are interested you can learn more at our website.

03 Dec 2013 1:29pm GMT

SimPy is a discrete event-simulation library for Python. Recently, the new version 3 has ben released. With that new release, we also changed the structure of the documentation. It now features a Tutorial, Topical Guides, an API Reference and a list of examples. Most of the topical guides have yet to written, though. So this is the first in a series of posts describing various concepts of SimPy.

If you break SimPy down, it is just an asynchronous event dispatcher. You generate events and schedule them at a given simulation time. Events are sorted by priority, simulation time, and an increasing event id. An event also has a list of callbacks, which are executed when the event is triggered and processed by the event loop. Events may also have a return value.

The components involved in this are the Environment, events and the process functions that you write.

Process functions implement your simulation model, that is, they define the behavior of your simulation. They are plain Python generator functions that yield instances of Event.

The environment stores these events in its event list and keeps track of the current simulation time.

If a process function yields and event, SimPy adds the process to the event's callbacks and suspends the process until the event is triggered and processed. When a process waiting for an event is resumed, it will also receive the event's value.

Here is a very simple example that illustrates all this; the code is more verbose than it needs to be to make things extra clear. You find a compact version of it at the end of this section:

>>> import simpy
>>>
>>> def example(env):
...     event = simpy.events.Timeout(env, delay=1, value=42)
...     value = yield event
...     print('now=%d, value=%d' % (env.now, value))
>>>
>>> env = simpy.Environment()
>>> example_gen = example(env)
>>> p = simpy.events.Process(env, example_gen)
>>>
>>> env.run()
now=1, value=42

The example() process function above first creates a Timeout event. It passes the environment, a delay, and a value to it. The Timeout schedules itself at now + delay (that's why the environment is required); other event types usually schedule themselves at the current simulation time.

The process function then yields the event and thus gets suspended. It is resumed, when SimPy processes the Timeout event. The process function also receives the event's value (42) - this is, however, optional, so yield event would have been okay if the you were not interested in the value or if the event had no value at all.

Finally, the process function prints the current simulation time (that is accessible via the environment's now attribute) and the Timeout's value.

If all required process functions are defined, you can instantiate all objects for your simulation. In most cases, you start by creating an instance of Environment, because you'll need to pass it around a lot when creating everything else.

Starting a process function involves two things:

1. You have to call the process function to create a generator object. (This will not execute any code of that function yet. Please read The Python yield keyword explained, to understand why this is the case.)
2. You then create an instance of Process and pass the environment and the generator object to it. This will schedule an Initialize event at the current simulation time which starts the execution of the process function. The process instance is also an event that is triggered when the process function returns. The guide to events (not yet written) explains why this is handy.

Finally, you can start SimPy's event loop. By default, it will run as long as there are events in the event list, but you can also let it stop earlier by providing an until argument.

The next guide/post will describe the environment and its interactions with events and process functions in more detail.

>>> import simpy
>>>
>>> def example(env):
...     value = yield env.timeout(1, value=42)
...     print('now=%d, value=%d' % (env.now, value))
>>>
>>> env = simpy.Environment()
>>> p = env.process(example(env))
>>> env.run()
now=1, value=42

03 Dec 2013 8:48am GMT

SimPy is a discrete event-simulation library for Python. Recently, the new version 3 has ben released. With that new release, we also changed the structure of the documentation. It now features a Tutorial, Topical Guides, an API Reference and a list of examples. Most of the topical guides have yet to written, though. So this is the first in a series of posts describing various concepts of SimPy.

If you break SimPy down, it is just an asynchronous event dispatcher. You generate events and schedule them at a given simulation time. Events are sorted by priority, simulation time, and an increasing event id. An event also has a list of callbacks, which are executed when the event is triggered and processed by the event loop. Events may also have a return value.

The components involved in this are the Environment, events and the process functions that you write.

Process functions implement your simulation model, that is, they define the behavior of your simulation. They are plain Python generator functions that yield instances of Event.

The environment stores these events in its event list and keeps track of the current simulation time.

If a process function yields and event, SimPy adds the process to the event's callbacks and suspends the process until the event is triggered and processed. When a process waiting for an event is resumed, it will also receive the event's value.

Here is a very simple example that illustrates all this; the code is more verbose than it needs to be to make things extra clear. You find a compact version of it at the end of this section:

>>> import simpy
>>>
>>> def example(env):
...     event = simpy.events.Timeout(env, delay=1, value=42)
...     value = yield event
...     print('now=%d, value=%d' % (env.now, value))
>>>
>>> env = simpy.Environment()
>>> example_gen = example(env)
>>> p = simpy.events.Process(env, example_gen)
>>>
>>> env.run()
now=1, value=42

The example() process function above first creates a Timeout event. It passes the environment, a delay, and a value to it. The Timeout schedules itself at now + delay (that's why the environment is required); other event types usually schedule themselves at the current simulation time.

The process function then yields the event and thus gets suspended. It is resumed, when SimPy processes the Timeout event. The process function also receives the event's value (42) - this is, however, optional, so yield event would have been okay if the you were not interested in the value or if the event had no value at all.

Finally, the process function prints the current simulation time (that is accessible via the environment's now attribute) and the Timeout's value.

If all required process functions are defined, you can instantiate all objects for your simulation. In most cases, you start by creating an instance of Environment, because you'll need to pass it around a lot when creating everything else.

Starting a process function involves two things:

1. You have to call the process function to create a generator object. (This will not execute any code of that function yet. Please read The Python yield keyword explained, to understand why this is the case.)
2. You then create an instance of Process and pass the environment and the generator object to it. This will schedule an Initialize event at the current simulation time which starts the execution of the process function. The process instance is also an event that is triggered when the process function returns. The guide to events (not yet written) explains why this is handy.

Finally, you can start SimPy's event loop. By default, it will run as long as there are events in the event list, but you can also let it stop earlier by providing an until argument.

The next guide/post will describe the environment and its interactions with events and process functions in more detail.

>>> import simpy
>>>
>>> def example(env):
...     value = yield env.timeout(1, value=42)
...     print('now=%d, value=%d' % (env.now, value))
>>>
>>> env = simpy.Environment()
>>> p = env.process(example(env))
>>> env.run()
now=1, value=42

03 Dec 2013 8:48am GMT

Update: We just added Christian O'Reilly as a speaker

As both the year and our alphabet reach their end, we are organizing again an amazing Python meetup evening, just before the holiday season. Our friends of w.illi.am/ are graceful enough to host our event at their offices near UQAM and Benelux.

We are hoping that this edition will allow you to get an answer to life, the universe (42), and everything in between [1], either way it should be fun !

Lightning talks:

• Rory Geoghegan - Module of the Month: csv
• Éric Araujo - Person of the Month

Speakers:

• Pablo Duboue - Introduction à Lamson plus the poisson magique play-by-email RPG gateway

• Simon Charette - Comment contribuer à Django

• Christian O'Reilly - Outils pythons pour l'analyse scientifique de l'électroencéphalogramme (EEG)

Cette présentation fera un bref survole de quatre projets à code source ouvert écrits en python et voués à l'investigation scientifique de l'EEG : Spyndle, BlockWork, Analyzer et Navigator. Ces outils sont complémentaires et forment ensemble une plateforme bien adaptée pour le traitement distribué de volumes importants de données.

• Marc-Antoine Ruel - Construire Google Chrome, un changement à la fois:

Google Chrome a 5 ans et est rendu à la version 31. Chaque nouvelle version contient au delà de 7000 changements.

Venez écouter comment une infrastructure écrite en python peut supporter une armée de développeurs, contenant une bonne proportion de contributeurs externes à Google, qui veulent intégrer des changements à toute heure du jour et de la nuit.

When:

Monday, the 9th of December 2013

Where:

w.illi.am/ offices 400, boul. de Maisonneuve Ouest, bureau 700 (http://goo.gl/maps/035B0)

Schedule:

• 6:00pm - Doors open
• 6:30pm - Presentations start
• 8:15pm - Break
• 8:30pm - Second round of presentations
• 10:00pm - One free beer offered at Bénélux just across the street

We'd like to thank our sponsors for their continuous support:

• UQÀM
• Bénélux
• w.illi.am/
• Outbox
• Savoir-Faire Linux
• Caravan
• iWeb

[1] - Wikipedia, Answer to the Ultimate Question of Life, the Universe, and Everything (42)

03 Dec 2013 12:37am GMT

Update: We just added Christian O'Reilly as a speaker

As both the year and our alphabet reach their end, we are organizing again an amazing Python meetup evening, just before the holiday season. Our friends of w.illi.am/ are graceful enough to host our event at their offices near UQAM and Benelux.

We are hoping that this edition will allow you to get an answer to life, the universe (42), and everything in between [1], either way it should be fun !

Lightning talks:

• Rory Geoghegan - Module of the Month: csv
• Éric Araujo - Person of the Month

Speakers:

• Pablo Duboue - Introduction à Lamson plus the poisson magique play-by-email RPG gateway

• Simon Charette - Comment contribuer à Django

• Christian O'Reilly - Outils pythons pour l'analyse scientifique de l'électroencéphalogramme (EEG)

Cette présentation fera un bref survole de quatre projets à code source ouvert écrits en python et voués à l'investigation scientifique de l'EEG : Spyndle, BlockWork, Analyzer et Navigator. Ces outils sont complémentaires et forment ensemble une plateforme bien adaptée pour le traitement distribué de volumes importants de données.

• Marc-Antoine Ruel - Construire Google Chrome, un changement à la fois:

Google Chrome a 5 ans et est rendu à la version 31. Chaque nouvelle version contient au delà de 7000 changements.

Venez écouter comment une infrastructure écrite en python peut supporter une armée de développeurs, contenant une bonne proportion de contributeurs externes à Google, qui veulent intégrer des changements à toute heure du jour et de la nuit.

When:

Monday, the 9th of December 2013

Where:

w.illi.am/ offices 400, boul. de Maisonneuve Ouest, bureau 700 (http://goo.gl/maps/035B0)

Schedule:

• 6:00pm - Doors open
• 6:30pm - Presentations start
• 8:15pm - Break
• 8:30pm - Second round of presentations
• 10:00pm - One free beer offered at Bénélux just across the street

We'd like to thank our sponsors for their continuous support:

• UQÀM
• Bénélux
• w.illi.am/
• Outbox
• Savoir-Faire Linux
• Caravan
• iWeb

[1] - Wikipedia, Answer to the Ultimate Question of Life, the Universe, and Everything (42)

03 Dec 2013 12:37am GMT

["Building web applications with Flask"]

03 Dec 2013 12:00am GMT

["Building web applications with Flask"]

03 Dec 2013 12:00am GMT

Yet another Monday, but the day was different. After waking up, I was not in a hurry to take bath and fix my laptop bag, instead took time to keep looking from the window to the outside. After more than 5 years, I was not in a rush to run to the office, instead the office came to my house.

I started the system to login to the IRC for my new job, in the community team of Eucalyptus. After the initial setups were done, I went ahead and started watching few recorded talks.

I am already running Eucalyptus in my home network for a private cloud for some time, where I test Fedora cloud images, and keep doing all random experiments. I will write about them in next few blog posts. Having centos instances also helped to test the Python codes in Python 2.6 series.

My primary job responsibility will be the community, to learn and grow with everyone else, to make sure that people can build the private cloud, they always wanted.

02 Dec 2013 8:41pm GMT

Yet another Monday, but the day was different. After waking up, I was not in a hurry to take bath and fix my laptop bag, instead took time to keep looking from the window to the outside. After more than 5 years, I was not in a rush to run to the office, instead the office came to my house.

I started the system to login to the IRC for my new job, in the community team of Eucalyptus. After the initial setups were done, I went ahead and started watching few recorded talks.

I am already running Eucalyptus in my home network for a private cloud for some time, where I test Fedora cloud images, and keep doing all random experiments. I will write about them in next few blog posts. Having centos instances also helped to test the Python codes in Python 2.6 series.

My primary job responsibility will be the community, to learn and grow with everyone else, to make sure that people can build the private cloud, they always wanted.

02 Dec 2013 8:41pm GMT

I've joined forces with Matt Harrison and Audrey Roy to put together the first ever Python Indie Bundle sale!

This bundle includes the following e-book bundles (PDF, Kindle, ePub) for just $24.95:

• Treading on Python Volume 1: Foundations
• Treading on Python Volume 2: Intermediate
• Two Scoops of Django: Best Practices for Django 1.5

Bought individually, these three books would cost you $36.95, but in honor of Cyber Monday we've chopped off about 33%, bringing you the awesome price of just $24.95!

We're just indie authors who work without a publisher. We pay all the bills, do all the marketing, and make it or break it on our own. It's a lot of work, but we think the total control we have over our work speaks volumes for the quality of result.

If you've been to my blog before, you probably already know about Two Scoops of Django that I co-wrote with Audrey Roy. However, you may not know about Matt Harrison's books, so I'm going to share some thoughts:

Matt has a talent for explaining things related to Python. If you are still new to Python, his books will amplify your skills and get you to the next level. If you've been doing Python for a while, his intermediate book will illuminate how things like decorators, generators, and other great features of Python can be leveraged to make your code cleaner and faster. In a nutshell, Matt's books pay for themselves because they ramp up your skills.

02 Dec 2013 6:46pm GMT

I've joined forces with Matt Harrison and Audrey Roy to put together the first ever Python Indie Bundle sale!

This bundle includes the following e-book bundles (PDF, Kindle, ePub) for just $24.95:

• Treading on Python Volume 1: Foundations
• Treading on Python Volume 2: Intermediate
• Two Scoops of Django: Best Practices for Django 1.5

Bought individually, these three books would cost you $36.95, but in honor of Cyber Monday we've chopped off about 33%, bringing you the awesome price of just $24.95!

We're just indie authors who work without a publisher. We pay all the bills, do all the marketing, and make it or break it on our own. It's a lot of work, but we think the total control we have over our work speaks volumes for the quality of result.

If you've been to my blog before, you probably already know about Two Scoops of Django that I co-wrote with Audrey Roy. However, you may not know about Matt Harrison's books, so I'm going to share some thoughts:

Matt has a talent for explaining things related to Python. If you are still new to Python, his books will amplify your skills and get you to the next level. If you've been doing Python for a while, his intermediate book will illuminate how things like decorators, generators, and other great features of Python can be leveraged to make your code cleaner and faster. In a nutshell, Matt's books pay for themselves because they ramp up your skills.

02 Dec 2013 6:46pm GMT

Recently I've started learning about Kivy, a Python Natural User Interface (NUI) toolkit. As I understand it, Kivy is kind of a spiritual successor to pyMT, which you can read more about here. In this article, we will be learning how Kivy handles layout management. While you can position widgets using x/y coordinates, in every GUI toolkit I've used, it's almost always better to use some kind of layout management that the toolkit provides. This allows the widgets to resize and move appropriately as the user changes the window's size. In Kivy, these things Layouts. If you've used wxPython, they are analogous to wxPython's sizers.

I should also note that Kivy can do layouts in two different ways. The first way is to do Layouts with Python code only. The second way is to use a mixture of Python and Kv language. This is to promote the model-view-controller way of doing things. It looks kind of like CSS and reminds me a little of wxPython and XRC. We will look at how to use both methods in this article. While Kivy supports multiple types of Layouts, this article will be focusing only on the BoxLayout. We will show how to nest BoxLayouts.

Kivy, Python and BoxLayout

Creating a BoxLayout in Kivy using Python is actually pretty easy and quite intuitive. We'll start out with a code example and then follow the code with an explanation. Let's get started!

import kivy
import random

from kivy.app import App
from kivy.uix.button import Button
from kivy.uix.boxlayout import BoxLayout

red = [1,0,0,1]
green = [0,1,0,1]
blue =  [0,0,1,1]
purple = [1,0,1,1]

########################################################################
class HBoxLayoutExample(App):
    """
    Horizontally oriented BoxLayout example class
    """

    #----------------------------------------------------------------------
    def build(self):
        """
        Horizontal BoxLayout example
        """
        layout = BoxLayout(padding=10)
        colors = [red, green, blue, purple]

        for i in range(5):
            btn = Button(text="Button #%s" % (i+1),
                         background_color=random.choice(colors)
                         )

            layout.add_widget(btn)
        return layout

########################################################################
class VBoxLayoutExample(App):
    """
    Vertical oriented BoxLayout example class
    """

    #----------------------------------------------------------------------
    def setOrientation(self, orient):
        """"""
        self.orient = orient

    #----------------------------------------------------------------------
    def build(self):
        """"""
        layout = BoxLayout(padding=10, orientation=self.orient)

        for i in range(5):
            btn = Button(text="Button #%s" % (i+1) )
            layout.add_widget(btn)
        return layout

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = HBoxLayoutExample()
    #app = VBoxLayoutExample()
    #app.setOrientation(orient="vertical")
    app.run()

Here we have created a vertically oriented BoxLayout class and a horizontally oriented BoxLayout class. Each class contains 5 buttons with random background colors. The colors follow RGBA, but can have individual values that are between zero and one. Oddly enough, if you use numbers greater than one, the color becomes brighter. I happened to use 255 instead of 1 when I created the screenshot above, so if you happen to run this code and see a more muted set of colors, that's why.

To make the examples extremely simple, we only import Kivy's App, Button and BoxLayout classes. The BoxLayout class accepts several arguments, but we'll focus on the following 3: orientation, padding and spacing. Because BoxLayout is a sub-class of Layout and Widget, it inherits many other methods and keyword arguments that are not covered here. But back to the arguments we currently care about. The padding argument tells Kivy how much space there should be between the Layout and its children, whereas the spacing arguments tells it how much spacing there should be between the children.

To create the buttons, we use a simple loop that loops over a small range of numbers. Each iteration creates a button with a random background color and adds that button to the Layout instance. Then we return the layout at the end.

The vertical BoxLayout example in the VBoxLayoutExample class is slightly different in that I thought it would be fun to be able to set the orientation programmatically. The code is pretty much the same except that I added a setOrientation method. Note that if you call setOrientation again, it will have no effect. As one of my commenters so kindly pointed out, you would need to bind the orientation to the App orient property or use the Kv language to achieve this.

If you comment out the call to HBoxLayoutExample at the end of the script and un-comment out the other two lines, then you should end up seeing something like this:

Notice that when you don't set a background color, Kivy defaults to a dark grey. Kivy does not try to look like a native application. This may or may not be a big deal to you depending on what sort of program you're trying to achieve, but it should be noted. Now we're ready to learn about nesting!

Nesting BoxLayouts

Nesting BoxLayouts inside of each other is pretty easy with Kivy too. Whenever you go to create an application with a complicated interface that will need nested sizers, you should take some time to sketch the layout out with pencil and paper. Then you can draw boxes around the widgets in different ways to help you visualize which Layouts you'll need and how to nest them in each other. I have found this quite helpful with wxPython and I think it applies to any other GUI toolkit that doesn't have a WYSIWYG editor. By the way, BoxLayouts are very powerful. If you know what you're doing, you can make just about any interface with them alone just be using clever nesting.

Enough talk, let's look at some code!

import kivy
import random

from kivy.app import App
from kivy.uix.button import Button
from kivy.uix.boxlayout import BoxLayout

red = [1,0,0,1]
green = [0,1,0,1]
blue =  [0,0,1,1]
purple = [1,0,1,1]

########################################################################
class NestedLayoutExample(App):
    """
    An example of nesting three horizontally oriented BoxLayouts inside
    of one vertically oriented BoxLayout
    """

    #----------------------------------------------------------------------
    def build(self):
        """
        Horizontal BoxLayout example
        """
        main_layout = BoxLayout(padding=10, orientation="vertical")

        colors = [red, green, blue, purple]

        for i in range(3):
            h_layout = BoxLayout(padding=10)
            for i in range(5):
                btn = Button(text="Button #%s" % (i+1),
                             background_color=random.choice(colors)
                             )

                h_layout.add_widget(btn)
            main_layout.add_widget(h_layout)

        return main_layout

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = NestedLayoutExample()
    app.run()

This example is very similar to the last one. The devil is in the details though. Here we have a nested for loop that creates 3 BoxLayouts that contain 5 buttons a piece. Each Layout is then inserted into the top level Layout at the end of each iteration in the outside loop. In case you missed it, scroll back up to see how it turned out. The trick is to create one top-level or main Layout and add other Layouts to it. Now let's turn our attention to learning how to do these things with the Kv language.

Kv+Python and BoxLayout

It's almost always a little painful to learn a new language. Fortunately, the Kv language actually follows Python pretty closely, including Python's requirement of using indentation levels to denote when a section of code begins and ends. You may want to spend a few minutes reading about the Kv language on the Kivy website. Whenever you're ready, we can continue. First we'll start off with the Python code:

# kvboxlayout.py

from kivy.app import App
from kivy.uix.boxlayout import BoxLayout

########################################################################
class KVMyHBoxLayout(BoxLayout):
    pass

########################################################################
class KVBoxLayoutApp(App):
    """"""

    #----------------------------------------------------------------------
    def build(self):
        """"""
        return KVMyHBoxLayout()

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = KVBoxLayoutApp()
    app.run()

This code is much simpler than our previous examples, but it's also rather mysterious. First of all, we create an empty sub-class of BoxLayout. Then we create our App class which has a build method that just returns an instance of the empty BoxLayout class. What's going on here? Well we have to look at the Kv file to find out!

<MyButton@Button>:
    color: .8,.9,0,1
    font_size: 32

<MyButton@Button>:
    orientation: 'horizontal'
    MyButton:
        text: "Btn1"
        background_color: 1,0,0,1
    MyButton:
        text: "Btn2"
        background_color: 0,1,0,1
    MyButton:
        text: "Btn3"
        background_color: 0,0,1,1
    MyButton:
        text: "Btn4"
        background_color: 1,0,1,1
    MyButton:
        text: "Btn5"
        background_color: 1,0,0,1

When you save the code above, you'll have to name it to be the same as the App class, but with a .kv instead of a .py and in lowercase. That means the name of this Kv file needs to be kvboxlayout.kv. You will note that you also need to strip off the App part of the class name such that KVBoxLayoutApp becomes kvboxlayout. Yes, it's a little confusing. If you don't follow the naming conventions correctly, the file will run but you will have an empty black window.

Anyway, first off in the Kv file, we have a section that starts with :. This tells Kivy that we are sub-classing the Button class and calling our sub-class MyButton. Then we indent the required four spaces and set the button's label color and font size. Next we create a BoxLayout section. Notice that we didn't create a sub-class this time. Then we tell it what orientation it should be and add 5 MyButton instances, each one having its own individual label and color.

One of the core Kivy developers pointed out that by creating the BoxLayout in this manner, I am redefining the BoxLayout for all usages. This is not a good thing, even if it does make the example simpler. Thus in the next example, we'll stop doing that and do it the right way instead!

Nesting BoxLayouts with Kv

Nesting BoxLayouts in Kv is a little confusing at first, but once you get the hang of it, you'll find that it's really quite easy. We'll start out with the Python code, take a look at how it works and then look at the Kv code.

from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.widget import Widget

########################################################################
class HBoxWidget(Widget):
    pass

########################################################################
class VBoxWidget(Widget):
    pass

########################################################################
class KVNestedBoxLayoutApp(App):
    """"""

    #----------------------------------------------------------------------
    def build(self):
        """"""
        return VBoxWidget()

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = KVNestedBoxLayoutApp()
    app.run()

This time around, we need to create two generic Widget classes: HBoxWidget and VBoxWidget. These are actually dummy classes that become BoxLayouts in the Kv code. Speaking of which, let's take a look at that now. Note that you'll need to name the Kv file kvnestedboxlayout.kv, which you'll note, is a lowercase version of KVNestedBoxLayoutApp.

:
    color: .8,.9,0,1
    font_size: 32

:
    BoxLayout:
        size: root.size
        pos: root.pos
        orientation: 'horizontal'
        MyButton:
            text: "Btn1"
            background_color: 1,1,1,1
        MyButton:
            text: "Btn2"
            background_color: 0,1,0,1
        MyButton:
            text: "Btn3"
            background_color: 0,0,1,1
        MyButton:
            text: "Btn4"
            background_color: 1,0,1,1
        MyButton:
            text: "Btn2"
            background_color: 1,0,0,1

:
    BoxLayout:
        size: root.size
        pos: root.pos
        id: foo_bar
        orientation: 'vertical'
        HBoxWidget:
        HBoxWidget:

The button code is the same as before. Next we have the HBoxWidget which we define as a horizontal BoxLayout with 5 buttons in it. Then we create an instance of VBoxWidget that is a vertical BoxLayout, but this Layout contains two instances of the HBoxWidget. You'll note that in the Python code's build method, we are returning the VBoxWidget, so that's where the action is. If you remove those two HBoxWidget calls, the result will be an empty black window.

There is another way to use Kv files in Kivy. It is through the kivy.lang.Builder.load_file (or load_string) API, which gives you the ability to load Kv files without needing to remember to name the Kv file in some special way. You can read about the API on their website and see an example of it in action in the Kivy examples on github. The only caveat to using this method is that you need to be careful not to load the same file twice or your UI may get messed up.

Wrapping Up

This just scratches the surface of Kivy's Layout system. There are 6 other Layout types available. However, I think you'll find that the examples in this article will get you started down the road of successfully creating cool Kivy applications of your very own. If you need help learning Kivy, there's a pretty good set of documentation on their website. They also have a Google Group and a #kivy channel on freenode.

Related Readings

• Kivy's Getting Started with Layouts
• Kivy's programming guide also has coverage on layouts
• A simple nested Layout example on github
• Kivy's Kv language page
• Other Kivy examples on github

Download the Source

• kivy_box_layouts.tar
• kivy_box_layouts.zip

Note: This is an official cross-posting of an article from the Mouse Vs Python blog. You can read the original here.

02 Dec 2013 6:36pm GMT

Recently I've started learning about Kivy, a Python Natural User Interface (NUI) toolkit. As I understand it, Kivy is kind of a spiritual successor to pyMT, which you can read more about here. In this article, we will be learning how Kivy handles layout management. While you can position widgets using x/y coordinates, in every GUI toolkit I've used, it's almost always better to use some kind of layout management that the toolkit provides. This allows the widgets to resize and move appropriately as the user changes the window's size. In Kivy, these things Layouts. If you've used wxPython, they are analogous to wxPython's sizers.

I should also note that Kivy can do layouts in two different ways. The first way is to do Layouts with Python code only. The second way is to use a mixture of Python and Kv language. This is to promote the model-view-controller way of doing things. It looks kind of like CSS and reminds me a little of wxPython and XRC. We will look at how to use both methods in this article. While Kivy supports multiple types of Layouts, this article will be focusing only on the BoxLayout. We will show how to nest BoxLayouts.

Kivy, Python and BoxLayout

Creating a BoxLayout in Kivy using Python is actually pretty easy and quite intuitive. We'll start out with a code example and then follow the code with an explanation. Let's get started!

import kivy
import random

from kivy.app import App
from kivy.uix.button import Button
from kivy.uix.boxlayout import BoxLayout

red = [1,0,0,1]
green = [0,1,0,1]
blue =  [0,0,1,1]
purple = [1,0,1,1]

########################################################################
class HBoxLayoutExample(App):
    """
    Horizontally oriented BoxLayout example class
    """

    #----------------------------------------------------------------------
    def build(self):
        """
        Horizontal BoxLayout example
        """
        layout = BoxLayout(padding=10)
        colors = [red, green, blue, purple]

        for i in range(5):
            btn = Button(text="Button #%s" % (i+1),
                         background_color=random.choice(colors)
                         )

            layout.add_widget(btn)
        return layout

########################################################################
class VBoxLayoutExample(App):
    """
    Vertical oriented BoxLayout example class
    """

    #----------------------------------------------------------------------
    def setOrientation(self, orient):
        """"""
        self.orient = orient

    #----------------------------------------------------------------------
    def build(self):
        """"""
        layout = BoxLayout(padding=10, orientation=self.orient)

        for i in range(5):
            btn = Button(text="Button #%s" % (i+1) )
            layout.add_widget(btn)
        return layout

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = HBoxLayoutExample()
    #app = VBoxLayoutExample()
    #app.setOrientation(orient="vertical")
    app.run()

Here we have created a vertically oriented BoxLayout class and a horizontally oriented BoxLayout class. Each class contains 5 buttons with random background colors. The colors follow RGBA, but can have individual values that are between zero and one. Oddly enough, if you use numbers greater than one, the color becomes brighter. I happened to use 255 instead of 1 when I created the screenshot above, so if you happen to run this code and see a more muted set of colors, that's why.

To make the examples extremely simple, we only import Kivy's App, Button and BoxLayout classes. The BoxLayout class accepts several arguments, but we'll focus on the following 3: orientation, padding and spacing. Because BoxLayout is a sub-class of Layout and Widget, it inherits many other methods and keyword arguments that are not covered here. But back to the arguments we currently care about. The padding argument tells Kivy how much space there should be between the Layout and its children, whereas the spacing arguments tells it how much spacing there should be between the children.

To create the buttons, we use a simple loop that loops over a small range of numbers. Each iteration creates a button with a random background color and adds that button to the Layout instance. Then we return the layout at the end.

The vertical BoxLayout example in the VBoxLayoutExample class is slightly different in that I thought it would be fun to be able to set the orientation programmatically. The code is pretty much the same except that I added a setOrientation method. Note that if you call setOrientation again, it will have no effect. As one of my commenters so kindly pointed out, you would need to bind the orientation to the App orient property or use the Kv language to achieve this.

If you comment out the call to HBoxLayoutExample at the end of the script and un-comment out the other two lines, then you should end up seeing something like this:

Notice that when you don't set a background color, Kivy defaults to a dark grey. Kivy does not try to look like a native application. This may or may not be a big deal to you depending on what sort of program you're trying to achieve, but it should be noted. Now we're ready to learn about nesting!

Nesting BoxLayouts

Nesting BoxLayouts inside of each other is pretty easy with Kivy too. Whenever you go to create an application with a complicated interface that will need nested sizers, you should take some time to sketch the layout out with pencil and paper. Then you can draw boxes around the widgets in different ways to help you visualize which Layouts you'll need and how to nest them in each other. I have found this quite helpful with wxPython and I think it applies to any other GUI toolkit that doesn't have a WYSIWYG editor. By the way, BoxLayouts are very powerful. If you know what you're doing, you can make just about any interface with them alone just be using clever nesting.

Enough talk, let's look at some code!

import kivy
import random

from kivy.app import App
from kivy.uix.button import Button
from kivy.uix.boxlayout import BoxLayout

red = [1,0,0,1]
green = [0,1,0,1]
blue =  [0,0,1,1]
purple = [1,0,1,1]

########################################################################
class NestedLayoutExample(App):
    """
    An example of nesting three horizontally oriented BoxLayouts inside
    of one vertically oriented BoxLayout
    """

    #----------------------------------------------------------------------
    def build(self):
        """
        Horizontal BoxLayout example
        """
        main_layout = BoxLayout(padding=10, orientation="vertical")

        colors = [red, green, blue, purple]

        for i in range(3):
            h_layout = BoxLayout(padding=10)
            for i in range(5):
                btn = Button(text="Button #%s" % (i+1),
                             background_color=random.choice(colors)
                             )

                h_layout.add_widget(btn)
            main_layout.add_widget(h_layout)

        return main_layout

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = NestedLayoutExample()
    app.run()

This example is very similar to the last one. The devil is in the details though. Here we have a nested for loop that creates 3 BoxLayouts that contain 5 buttons a piece. Each Layout is then inserted into the top level Layout at the end of each iteration in the outside loop. In case you missed it, scroll back up to see how it turned out. The trick is to create one top-level or main Layout and add other Layouts to it. Now let's turn our attention to learning how to do these things with the Kv language.

Kv+Python and BoxLayout

It's almost always a little painful to learn a new language. Fortunately, the Kv language actually follows Python pretty closely, including Python's requirement of using indentation levels to denote when a section of code begins and ends. You may want to spend a few minutes reading about the Kv language on the Kivy website. Whenever you're ready, we can continue. First we'll start off with the Python code:

# kvboxlayout.py

from kivy.app import App
from kivy.uix.boxlayout import BoxLayout

########################################################################
class KVMyHBoxLayout(BoxLayout):
    pass

########################################################################
class KVBoxLayoutApp(App):
    """"""

    #----------------------------------------------------------------------
    def build(self):
        """"""
        return KVMyHBoxLayout()

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = KVBoxLayoutApp()
    app.run()

This code is much simpler than our previous examples, but it's also rather mysterious. First of all, we create an empty sub-class of BoxLayout. Then we create our App class which has a build method that just returns an instance of the empty BoxLayout class. What's going on here? Well we have to look at the Kv file to find out!

<MyButton@Button>:
    color: .8,.9,0,1
    font_size: 32

<MyButton@Button>:
    orientation: 'horizontal'
    MyButton:
        text: "Btn1"
        background_color: 1,0,0,1
    MyButton:
        text: "Btn2"
        background_color: 0,1,0,1
    MyButton:
        text: "Btn3"
        background_color: 0,0,1,1
    MyButton:
        text: "Btn4"
        background_color: 1,0,1,1
    MyButton:
        text: "Btn5"
        background_color: 1,0,0,1

When you save the code above, you'll have to name it to be the same as the App class, but with a .kv instead of a .py and in lowercase. That means the name of this Kv file needs to be kvboxlayout.kv. You will note that you also need to strip off the App part of the class name such that KVBoxLayoutApp becomes kvboxlayout. Yes, it's a little confusing. If you don't follow the naming conventions correctly, the file will run but you will have an empty black window.

Anyway, first off in the Kv file, we have a section that starts with :. This tells Kivy that we are sub-classing the Button class and calling our sub-class MyButton. Then we indent the required four spaces and set the button's label color and font size. Next we create a BoxLayout section. Notice that we didn't create a sub-class this time. Then we tell it what orientation it should be and add 5 MyButton instances, each one having its own individual label and color.

One of the core Kivy developers pointed out that by creating the BoxLayout in this manner, I am redefining the BoxLayout for all usages. This is not a good thing, even if it does make the example simpler. Thus in the next example, we'll stop doing that and do it the right way instead!

Nesting BoxLayouts with Kv

Nesting BoxLayouts in Kv is a little confusing at first, but once you get the hang of it, you'll find that it's really quite easy. We'll start out with the Python code, take a look at how it works and then look at the Kv code.

from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.widget import Widget

########################################################################
class HBoxWidget(Widget):
    pass

########################################################################
class VBoxWidget(Widget):
    pass

########################################################################
class KVNestedBoxLayoutApp(App):
    """"""

    #----------------------------------------------------------------------
    def build(self):
        """"""
        return VBoxWidget()

#----------------------------------------------------------------------
if __name__ == "__main__":
    app = KVNestedBoxLayoutApp()
    app.run()

This time around, we need to create two generic Widget classes: HBoxWidget and VBoxWidget. These are actually dummy classes that become BoxLayouts in the Kv code. Speaking of which, let's take a look at that now. Note that you'll need to name the Kv file kvnestedboxlayout.kv, which you'll note, is a lowercase version of KVNestedBoxLayoutApp.

:
    color: .8,.9,0,1
    font_size: 32

:
    BoxLayout:
        size: root.size
        pos: root.pos
        orientation: 'horizontal'
        MyButton:
            text: "Btn1"
            background_color: 1,1,1,1
        MyButton:
            text: "Btn2"
            background_color: 0,1,0,1
        MyButton:
            text: "Btn3"
            background_color: 0,0,1,1
        MyButton:
            text: "Btn4"
            background_color: 1,0,1,1
        MyButton:
            text: "Btn2"
            background_color: 1,0,0,1

:
    BoxLayout:
        size: root.size
        pos: root.pos
        id: foo_bar
        orientation: 'vertical'
        HBoxWidget:
        HBoxWidget:

The button code is the same as before. Next we have the HBoxWidget which we define as a horizontal BoxLayout with 5 buttons in it. Then we create an instance of VBoxWidget that is a vertical BoxLayout, but this Layout contains two instances of the HBoxWidget. You'll note that in the Python code's build method, we are returning the VBoxWidget, so that's where the action is. If you remove those two HBoxWidget calls, the result will be an empty black window.

There is another way to use Kv files in Kivy. It is through the kivy.lang.Builder.load_file (or load_string) API, which gives you the ability to load Kv files without needing to remember to name the Kv file in some special way. You can read about the API on their website and see an example of it in action in the Kivy examples on github. The only caveat to using this method is that you need to be careful not to load the same file twice or your UI may get messed up.

Wrapping Up

This just scratches the surface of Kivy's Layout system. There are 6 other Layout types available. However, I think you'll find that the examples in this article will get you started down the road of successfully creating cool Kivy applications of your very own. If you need help learning Kivy, there's a pretty good set of documentation on their website. They also have a Google Group and a #kivy channel on freenode.

Related Readings

• Kivy's Getting Started with Layouts
• Kivy's programming guide also has coverage on layouts
• A simple nested Layout example on github
• Kivy's Kv language page
• Other Kivy examples on github

Download the Source

• kivy_box_layouts.tar
• kivy_box_layouts.zip

Note: This is an official cross-posting of an article from the Mouse Vs Python blog. You can read the original here.

02 Dec 2013 6:36pm GMT

Zero to hero

What Prompted me??

I started my third year thinking I should do something that would put me different from the rest and one of my professors suggested me as to why don't I apply for GSoC. I don't know why but I took the suggestion rather seriously, thanks to the bet I had with one of my friend(who is about to complete his MBBS) that whoever earns first will buy the other a "RayBan shades". Well, that's it. I was determined. I started my research early, probably during the start of February(I knew I want to buy my friend, his shades and also buy mine too, in the process).

What experiences I had before??

I started looking at previous years' GSoC projects(having had little experience with Open Source) and started learning how to contribute. I was also very fascinated to the amount of knowledge one could gain just by googling and browsing web pages . I discovered very soon, as to what an immensely great tool , email, through which I could chat with anyone in the open source world and ask seemingly stupid questions and always expect to get a gentle reply back with an answer. Well, that held me spell bound and I knew I want to contribute to Open Source.

How did I begin??

About the middle of March, I discovered that my passion for Python as a programming language increased , after understanding how easy it is as a language. Added to that, my popularity among my fellow classmates increased when I started evangelizing Python(thanks to my seniors for introducing it, I guess I did a decent job popularizing the language). And I started contributing to PSF(Python Software Foundation) , started with a simple bug to fix documentation and slowly my interactivity in IRC increased and I started liking one of the project one of the community member proposed.

A twist in the story??

There I was, still a noob and not knowing how to convince my probable mentor that I could complete the project, given direction. About this juncture, a fellow student(from some university in France) mailed this particular mentor that he was interested in the project . Do, remember, I was part of the mailing list and follow the happenings of it. So, I was furious knowing that I had a competition(having put so much effort) and I was not willing to compromise my project (knowing that this is the one project I actually understood and started researching a little bit too). The other projects require me to have some domain knowledge. I went back to my teachers, seniors, friends and Google and started asking the question , "how would i solve the problem the mentor posted?" . I framed a couple of answers, though very noobish , but at least I could reply the email thread posting my understanding of the problem and how I would solve it and also ask various questions I had in my mind. Well, the mentor replied, immediately to my surprise, and responded back with comments as well as answers to the questions I posed. Again, my nemesis/competitor replied back(he having good knowledge about the problem domain). I knew it was not going to be easy. Hence, I went back again, through all my sources, made further understanding of the problem and posted back again. I guess, about 20 mails in the thread , till we(all three of us) decided we should catch up in IRC and discuss more.

The conclusion:

Well, at IRC , most of senior members from the community were present, and they suggested that they should probably extend the scope of the project(since two students were interested in one project and showed immense passion). Unsurprisingly, over multiple meetings, the project scope was expanded, both the students were given equal important but independent tasks and both the students got opportunity to say they are Google Summer of Code students. Thank goodness, we decided to built the project from scratch giving us more than enough work on our plate.

Movie titles:

1) In the open source world, there is no competition , it is only "COLLABORATION".

2) Why give up, when you can win??

3) From Zero to Hero!!

4) A prodigy in making

p.s. I still owe my friend his shades . *sshole, I am still waiting for him to visit me so that I can buy him his shades and buy mine too. Also, I know its been two years since the story happened, but it is never too late to share, don't you agree??

11 Oct 2013 5:39am GMT