fidzu : Python

Here are some stories on how you can go wrong with naming, especially in open source software.

29 Jul 2014 2:37pm GMT

Here are some stories on how you can go wrong with naming, especially in open source software.

29 Jul 2014 2:37pm GMT

When working with shell scripts written in bash/csh/etc. one of the primary tools you have to rely on is a simple method of piping output and input from subprocesses called by the script to create complex logic to accomplish the goal of the script. When working with python, this same method of calling subprocesses to redirect the input/output is available, but the overhead of using this method in python would be so cumbersome as to make python a less desirable scripting language. In effect you were implementing large parts of the I/O facilities, and potentially even writing replacements for the existing shell utilities that would perform the same work. Recently, python developers attempted to solve this problem, by updating an existing python subprocess wrapper library called pbs, into an easier to use library called sh.

Sh can be installed using pip, and the author has posted some documentation for the library here: http://amoffat.github.io/sh/

Using the sh library

After installing the library into your version of python, there will be two ways to call any existing shell command available to the system, firstly you can import the command as though it was itself a python library:

from sh import hostname
print(hostname())

In addition, you can also call the command directly by just referencing the sh namespace prior to the command name:

import sh
print(sh.hostname())

When running this command on my linux workstation (hostname atlas) it will return the expected results:

Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(sh.hostname())
atlas

However at this point, we are merely replacing a single shell command which prints output to the screen, the real benefit of the shell scripts was that you could chain together commands in order to create complex logic to help you do work.

Advanced Gymnastics

A common use of shell scripts is to provide administrators the ability to quickly filter log file output and to potentially search for specific conditions within those logs, to alert in the event that an application starts throwing errors. With python piping in sh we can create a simple log watcher, which would be capable of calling anything we desire in python when the log file contains any of the conditions we are looking for.

To pipe together commands using the sh library, you would encapsulate each command in series to create a similar syntax to bash piping:

>>> print(sh.wc(sh.ls("-l", "/etc"), "-l"))
199

This command would have been equivalent to the bash pipe of "ls -l /etc | wc -l" indicating that the long listing of /etc on my workstation contained 199 lines of output. Each piped command is encapsulated inside the parenthesis of the command the precedes it.

For our log listener we will use the tail command along with a python iterator to watch for a potential error condition, which I will represent with the string "ERROR":

>>> for line in sh.tail("-f", "/tmp/test_log", _iter=True):
...     if "ERROR" in line:
...         print line

In this example, once executed, python will call the tail command to follow a particular log file. It will iterate over each line of output produced by tail and if any of the lines contain the string we are watching for python will print that line to standard output. At this point, this would be similar to using the tail command and piping the output to a string search command, like grep. However, you could replace the third line of the python with a more complex action, emailing the error condition out to a developer or administrator for review, or perhaps initiating a procedure to recover from the error automatically.

Conclusions

In this manner with just a few lines of python, much like with bash, one could create a relatively complex process without recreating all the shell commands which already perform this work, or create a convoluted wrapping process of passing output from command to command. This combination of the existing shell commands and the power of python; you get all the functions available to any python environment, with the ease of using the shell commands to do some of the work. In the future I will definitely be using this python library for my own shell scripting needs, as I have generally preferred the syntax and ease of use of python over that of bash, but now I will be able to enjoy both at the same time.

29 Jul 2014 12:35pm GMT

When working with shell scripts written in bash/csh/etc. one of the primary tools you have to rely on is a simple method of piping output and input from subprocesses called by the script to create complex logic to accomplish the goal of the script. When working with python, this same method of calling subprocesses to redirect the input/output is available, but the overhead of using this method in python would be so cumbersome as to make python a less desirable scripting language. In effect you were implementing large parts of the I/O facilities, and potentially even writing replacements for the existing shell utilities that would perform the same work. Recently, python developers attempted to solve this problem, by updating an existing python subprocess wrapper library called pbs, into an easier to use library called sh.

Sh can be installed using pip, and the author has posted some documentation for the library here: http://amoffat.github.io/sh/

Using the sh library

After installing the library into your version of python, there will be two ways to call any existing shell command available to the system, firstly you can import the command as though it was itself a python library:

from sh import hostname
print(hostname())

In addition, you can also call the command directly by just referencing the sh namespace prior to the command name:

import sh
print(sh.hostname())

When running this command on my linux workstation (hostname atlas) it will return the expected results:

Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(sh.hostname())
atlas

However at this point, we are merely replacing a single shell command which prints output to the screen, the real benefit of the shell scripts was that you could chain together commands in order to create complex logic to help you do work.

Advanced Gymnastics

A common use of shell scripts is to provide administrators the ability to quickly filter log file output and to potentially search for specific conditions within those logs, to alert in the event that an application starts throwing errors. With python piping in sh we can create a simple log watcher, which would be capable of calling anything we desire in python when the log file contains any of the conditions we are looking for.

To pipe together commands using the sh library, you would encapsulate each command in series to create a similar syntax to bash piping:

>>> print(sh.wc(sh.ls("-l", "/etc"), "-l"))
199

This command would have been equivalent to the bash pipe of "ls -l /etc | wc -l" indicating that the long listing of /etc on my workstation contained 199 lines of output. Each piped command is encapsulated inside the parenthesis of the command the precedes it.

For our log listener we will use the tail command along with a python iterator to watch for a potential error condition, which I will represent with the string "ERROR":

>>> for line in sh.tail("-f", "/tmp/test_log", _iter=True):
...     if "ERROR" in line:
...         print line

In this example, once executed, python will call the tail command to follow a particular log file. It will iterate over each line of output produced by tail and if any of the lines contain the string we are watching for python will print that line to standard output. At this point, this would be similar to using the tail command and piping the output to a string search command, like grep. However, you could replace the third line of the python with a more complex action, emailing the error condition out to a developer or administrator for review, or perhaps initiating a procedure to recover from the error automatically.

Conclusions

In this manner with just a few lines of python, much like with bash, one could create a relatively complex process without recreating all the shell commands which already perform this work, or create a convoluted wrapping process of passing output from command to command. This combination of the existing shell commands and the power of python; you get all the functions available to any python environment, with the ease of using the shell commands to do some of the work. In the future I will definitely be using this python library for my own shell scripting needs, as I have generally preferred the syntax and ease of use of python over that of bash, but now I will be able to enjoy both at the same time.

29 Jul 2014 12:35pm GMT

On Part 1 we discussed the basics of embedding JavaScript code into an IPython Notebook, and saw how to use this feature to integrate D3.js charts. On this part we'll show you how to do the same with Chart.js.

Part 2 - Embedding ChartJS

First we need to declare the requirement using RequireJS:

%%javascript
require.config({
    paths: {
        chartjs: '//cdnjs.cloudflare.com/ajax/libs/Chart.js/0.2.0/Chart.min'
    }
});

The procedure is the same as before, we define a template that will contain the rendered JavaScript code, and we use display to embed the code into the running page.

Chart.js is an HTML5 charting library capable of producing beautiful graphics with very little code. Now we want to plot the male and female population by region, division an state. We'll use interact with a new callback function called display_chart_chartjs, but this time we don't need custom widgets as we're only selecting a single item (we'll use a DropdownWidget). We've also included the show_javascript checkbox.

i = interact(
    display_chart_chartjs,
    sc_est2012_sex=widgets.fixed(sc_est2012_sex),
    region=widgets.fixed(region),
    division=widgets.fixed(division),
    show_javascript=widgets.CheckboxWidget(value=False),
    show=widgets.DropdownWidget(
        values={'By Region':'by_region', 'By Division': 'by_division', 'By State': 'by_state'},
        value='by_region'
    ),
    div=widgets.HTMLWidget(value='<canvas width=800 height=400 id="chart_chartjs"></canvas>')
)

As you can see, the library generates an beautiful and simple animated column chart. There's not much in terms of customization of Chart.js charts, but that makes it very easy to use.

On the last part (soon), we'll show you how to embed HighCharts charts.

29 Jul 2014 12:01pm GMT

On Part 1 we discussed the basics of embedding JavaScript code into an IPython Notebook, and saw how to use this feature to integrate D3.js charts. On this part we'll show you how to do the same with Chart.js.

Part 2 - Embedding ChartJS

First we need to declare the requirement using RequireJS:

%%javascript
require.config({
    paths: {
        chartjs: '//cdnjs.cloudflare.com/ajax/libs/Chart.js/0.2.0/Chart.min'
    }
});

The procedure is the same as before, we define a template that will contain the rendered JavaScript code, and we use display to embed the code into the running page.

Chart.js is an HTML5 charting library capable of producing beautiful graphics with very little code. Now we want to plot the male and female population by region, division an state. We'll use interact with a new callback function called display_chart_chartjs, but this time we don't need custom widgets as we're only selecting a single item (we'll use a DropdownWidget). We've also included the show_javascript checkbox.

i = interact(
    display_chart_chartjs,
    sc_est2012_sex=widgets.fixed(sc_est2012_sex),
    region=widgets.fixed(region),
    division=widgets.fixed(division),
    show_javascript=widgets.CheckboxWidget(value=False),
    show=widgets.DropdownWidget(
        values={'By Region':'by_region', 'By Division': 'by_division', 'By State': 'by_state'},
        value='by_region'
    ),
    div=widgets.HTMLWidget(value='<canvas width=800 height=400 id="chart_chartjs"></canvas>')
)

As you can see, the library generates an beautiful and simple animated column chart. There's not much in terms of customization of Chart.js charts, but that makes it very easy to use.

On the last part (soon), we'll show you how to embed HighCharts charts.

29 Jul 2014 12:01pm GMT

PEP-396 suggests that all python packages should provide a __version__ attribute. This should contain a string in the form x.y.z which gives you the package's version number. That seems simple enough, except that Python does not provide an obvious way to set this attribute. It would be a tremendous waste of time to set this […]

29 Jul 2014 12:07am GMT

PEP-396 suggests that all python packages should provide a __version__ attribute. This should contain a string in the form x.y.z which gives you the package's version number. That seems simple enough, except that Python does not provide an obvious way to set this attribute. It would be a tremendous waste of time to set this […]

29 Jul 2014 12:07am GMT

If you are using Homebrew on OSX and Python and there is a small virtualenv then this article is for you.

After a small upgrade of Python via Brew, you risk to get this error when you want to use the virtualenv.

In my case, the directory where I store the virtualenvs is ~/.virtualenvs

$HOME/.virtualenvs/
    pelican/

Loads the virtualenv.

$HOME/.virtualenvs (0) > source ~/.virtualenvs/pelican/bin/activate.sh

$HOME/.virtualenvs (0) > (pelican) pelican
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: $HOME/.virtualenvs/pelican/bin/python
  Reason: image not found
fish: Job 1, 'pelican' terminated by signal SIGTRAP (Trace or breakpoint trap)

$HOME/.virtualenvs (0) > (pelican) deactivate

In fact, there is a problem with the library, you need to regenerate the links of the Python interpreter. In this case, this is just to remove the existing links and re-execute the virtualenv command.

$HOME/.virtualenvs (0) > find pelican/ -type l
pelican/.Python
pelican/bin/python
pelican/bin/python2
pelican/include/python2.7
pelican/lib/python2.7/_abcoll.py
pelican/lib/python2.7/_weakrefset.py
pelican/lib/python2.7/abc.py
pelican/lib/python2.7/codecs.py
pelican/lib/python2.7/config
pelican/lib/python2.7/copy_reg.py
pelican/lib/python2.7/encodings
pelican/lib/python2.7/fnmatch.py
pelican/lib/python2.7/genericpath.py
pelican/lib/python2.7/lib-dynload
pelican/lib/python2.7/linecache.py
pelican/lib/python2.7/locale.py
pelican/lib/python2.7/ntpath.py
pelican/lib/python2.7/os.py
pelican/lib/python2.7/posixpath.py
pelican/lib/python2.7/re.py
pelican/lib/python2.7/sre.py
pelican/lib/python2.7/sre_compile.py
pelican/lib/python2.7/sre_constants.py
pelican/lib/python2.7/sre_parse.py
pelican/lib/python2.7/stat.py
pelican/lib/python2.7/types.py
pelican/lib/python2.7/UserDict.py
pelican/lib/python2.7/warnings.py

All the symlinks are wrong, and we need to remove them and re-execute the virtualenv command.

$HOME/.virtualenvs (0) > find pelican/ -type l -delete

$HOME/.virtualenvs (0) > virtualenv pelican
Overwriting pelican/lib/python2.7/site.py with new content
Overwriting pelican/lib/python2.7/orig-prefix.txt with new content
New python executable in pelican/bin/python2.7
Also creating executable in pelican/bin/python
Installing setuptools, pip...done.
Overwriting pelican/bin/activate_this.py with new content

Loads the virtualenv ;-)

$HOME/.virtualenvs (0) > source ~/.virtualenvs/pelican/bin/activate.sh

$HOME/.virtualenvs > (pelican) pelican --version
3.5.dev

And now, you have a working virtualenv.

28 Jul 2014 10:00pm GMT

If you are using Homebrew on OSX and Python and there is a small virtualenv then this article is for you.

After a small upgrade of Python via Brew, you risk to get this error when you want to use the virtualenv.

In my case, the directory where I store the virtualenvs is ~/.virtualenvs

$HOME/.virtualenvs/
    pelican/

Loads the virtualenv.

$HOME/.virtualenvs (0) > source ~/.virtualenvs/pelican/bin/activate.sh

$HOME/.virtualenvs (0) > (pelican) pelican
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: $HOME/.virtualenvs/pelican/bin/python
  Reason: image not found
fish: Job 1, 'pelican' terminated by signal SIGTRAP (Trace or breakpoint trap)

$HOME/.virtualenvs (0) > (pelican) deactivate

In fact, there is a problem with the library, you need to regenerate the links of the Python interpreter. In this case, this is just to remove the existing links and re-execute the virtualenv command.

$HOME/.virtualenvs (0) > find pelican/ -type l
pelican/.Python
pelican/bin/python
pelican/bin/python2
pelican/include/python2.7
pelican/lib/python2.7/_abcoll.py
pelican/lib/python2.7/_weakrefset.py
pelican/lib/python2.7/abc.py
pelican/lib/python2.7/codecs.py
pelican/lib/python2.7/config
pelican/lib/python2.7/copy_reg.py
pelican/lib/python2.7/encodings
pelican/lib/python2.7/fnmatch.py
pelican/lib/python2.7/genericpath.py
pelican/lib/python2.7/lib-dynload
pelican/lib/python2.7/linecache.py
pelican/lib/python2.7/locale.py
pelican/lib/python2.7/ntpath.py
pelican/lib/python2.7/os.py
pelican/lib/python2.7/posixpath.py
pelican/lib/python2.7/re.py
pelican/lib/python2.7/sre.py
pelican/lib/python2.7/sre_compile.py
pelican/lib/python2.7/sre_constants.py
pelican/lib/python2.7/sre_parse.py
pelican/lib/python2.7/stat.py
pelican/lib/python2.7/types.py
pelican/lib/python2.7/UserDict.py
pelican/lib/python2.7/warnings.py

All the symlinks are wrong, and we need to remove them and re-execute the virtualenv command.

$HOME/.virtualenvs (0) > find pelican/ -type l -delete

$HOME/.virtualenvs (0) > virtualenv pelican
Overwriting pelican/lib/python2.7/site.py with new content
Overwriting pelican/lib/python2.7/orig-prefix.txt with new content
New python executable in pelican/bin/python2.7
Also creating executable in pelican/bin/python
Installing setuptools, pip...done.
Overwriting pelican/bin/activate_this.py with new content

Loads the virtualenv ;-)

$HOME/.virtualenvs (0) > source ~/.virtualenvs/pelican/bin/activate.sh

$HOME/.virtualenvs > (pelican) pelican --version
3.5.dev

And now, you have a working virtualenv.

28 Jul 2014 10:00pm GMT

So you have an audio stream where you'd like to get a human-friendly readout of the current audio level. You add a level component, but how do you actually get the level messages it generates?

            bus.add_signal_watch()
            bus.connect( 'message', self.on_level )

It really seems that you *should* be able to use element.connect(), but there doesn't seem to be an available event to which to connect on the level. So, you wind up having to process all of the events and look for your level event...

    def on_level( self, bus, message ):
        """Level message was received"""
        if message.src == self.monitor and message.type==gst.MESSAGE_ELEMENT:
            self.send( {
                'type':'level',
                'level': message.structure['rms'][0],
            })

Next problem, the "level" is not particularly human friendly. It appears to be a decibel attenuation (from 0 negative), where e.g. -17 seems to be a pretty loud voice and ~-40 is background noise... but that's just what *my* setup produces. Still trying to figure out if there's a formal "what pocketsphinx wants for dictation" definition somewhere. The "vader" element has a value of 0.0078125 as the default for volume cut-off, but no idea what that means.

28 Jul 2014 7:33pm GMT

So you have an audio stream where you'd like to get a human-friendly readout of the current audio level. You add a level component, but how do you actually get the level messages it generates?

            bus.add_signal_watch()
            bus.connect( 'message', self.on_level )

It really seems that you *should* be able to use element.connect(), but there doesn't seem to be an available event to which to connect on the level. So, you wind up having to process all of the events and look for your level event...

    def on_level( self, bus, message ):
        """Level message was received"""
        if message.src == self.monitor and message.type==gst.MESSAGE_ELEMENT:
            self.send( {
                'type':'level',
                'level': message.structure['rms'][0],
            })

Next problem, the "level" is not particularly human friendly. It appears to be a decibel attenuation (from 0 negative), where e.g. -17 seems to be a pretty loud voice and ~-40 is background noise... but that's just what *my* setup produces. Still trying to figure out if there's a formal "what pocketsphinx wants for dictation" definition somewhere. The "vader" element has a value of 0.0078125 as the default for volume cut-off, but no idea what that means.

28 Jul 2014 7:33pm GMT

PyOhio gave my lightning talk on ddlgenerator a warm reception, and Brandon Lorenz got me thinking, and PyOhio sprints filled my with py-drenaline, and now ddlgenerator can inspect your data and spit out SQLAlchemy model definitions for you:

$ cat merovingians.yaml 
-
  name: Clovis I
  reign:
    from: 486
    to: 511
-
  name: Childebert I
  reign:
    from: 511
    to: 558
$ ddlgenerator --inserts sqlalchemy merovingians.yaml 

from sqlalchemy import create_engine, Column, Integer, Table, Unicode
engine = create_engine(r'sqlite:///:memory:')
metadata = MetaData(bind=engine)

merovingians = Table('merovingians', metadata, 
  Column('name', Unicode(length=12), nullable=False), 
  Column('reign_from', Integer(), nullable=False), 
  Column('reign_to', Integer(), nullable=False), 
  schema=None)

metadata.create_all()
conn = engine.connect()
inserter = merovingians.insert()
conn.execute(inserter, **{'name': 'Clovis I', 'reign_from': 486, 'reign_to': 511})
conn.execute(inserter, **{'name': 'Childebert I', 'reign_from': 511, 'reign_to': 558})
conn.connection.commit()

Brandon's working on a pull request to provide similar functionality for Django models!

28 Jul 2014 3:30pm GMT

PyOhio gave my lightning talk on ddlgenerator a warm reception, and Brandon Lorenz got me thinking, and PyOhio sprints filled my with py-drenaline, and now ddlgenerator can inspect your data and spit out SQLAlchemy model definitions for you:

$ cat merovingians.yaml 
-
  name: Clovis I
  reign:
    from: 486
    to: 511
-
  name: Childebert I
  reign:
    from: 511
    to: 558
$ ddlgenerator --inserts sqlalchemy merovingians.yaml 

from sqlalchemy import create_engine, Column, Integer, Table, Unicode
engine = create_engine(r'sqlite:///:memory:')
metadata = MetaData(bind=engine)

merovingians = Table('merovingians', metadata, 
  Column('name', Unicode(length=12), nullable=False), 
  Column('reign_from', Integer(), nullable=False), 
  Column('reign_to', Integer(), nullable=False), 
  schema=None)

metadata.create_all()
conn = engine.connect()
inserter = merovingians.insert()
conn.execute(inserter, **{'name': 'Clovis I', 'reign_from': 486, 'reign_to': 511})
conn.execute(inserter, **{'name': 'Childebert I', 'reign_from': 511, 'reign_to': 558})
conn.connection.commit()

Brandon's working on a pull request to provide similar functionality for Django models!

28 Jul 2014 3:30pm GMT

The EP14 Pylint sprint team (more on this here and there) is proud to announce they just released Pylint 1.3 together with its companion Astroid 1.2. As usual, this includes several new features as well and bug fixes. You'll find below some structured list of the changes.

Packages are uploaded to pypi, debian/ubuntu packages should be soon provided by Logilab, until they get into the standard packaging system of your favorite distribution.

Please notice Pylint 1.3 will be the last release branch support python 2.5 and 2.6. Starting from 1.4, we will only support python greater or equal to 2.7. This will be the occasion to do some great cleanup in the code base. Notice this is only about the Pylint's runtime, you should still be able to run Pylint on your Python 2.5 code, through using Python 2.7 at least.

28 Jul 2014 3:21pm GMT

The EP14 Pylint sprint team (more on this here and there) is proud to announce they just released Pylint 1.3 together with its companion Astroid 1.2. As usual, this includes several new features as well and bug fixes. You'll find below some structured list of the changes.

Packages are uploaded to pypi, debian/ubuntu packages should be soon provided by Logilab, until they get into the standard packaging system of your favorite distribution.

Please notice Pylint 1.3 will be the last release branch support python 2.5 and 2.6. Starting from 1.4, we will only support python greater or equal to 2.7. This will be the occasion to do some great cleanup in the code base. Notice this is only about the Pylint's runtime, you should still be able to run Pylint on your Python 2.5 code, through using Python 2.7 at least.

28 Jul 2014 3:21pm GMT

Here are the list of things we managed to achieve during those last two days at EuroPython.

After several attempts, Michal managed to have pylint running analysis on several files in parallel. This is still in a pull request (https://bitbucket.org/logilab/pylint/pull-request/82/added-support-for-checking-files-in) because of some limitations, so we decided it won't be part of the 1.3 release.

Claudiu killed maybe 10 bugs or so and did some heavy issues cleanup in the trackers. He also demonstrated some experimental support of python 3 style annotation to drive a better inference. Pretty exciting! Torsten also killed several bugs, restored python 2.5 compat (though that will need a logilab-common release as well), introduced a new functional test framework that will replace the old one once all the existing tests will be backported. On wednesday, he did show us a near future feature they already have at Google: some kind of confidence level associated to messages so that you can filter out based on that. Sylvain fixed a couple of bugs (including https://bitbucket.org/logilab/pylint/issue/58/ which was annoying all the numpy community), started some refactoring of the PyLinter class so it does a little bit fewer things (still way too many though) and attempted to improve the pylint note on both pylint and astroid, which went down recently "thanks" to the new checks like 'bad-continuation'.

Also, we merged the pylint-brain project into astroid to simplify things, so you should now submit your brain plugins directly to the astroid project. Hopefuly you'll be redirected there on attempt to use the old (removed) pylint-brain project on bitbucket.

And, the good news is that now both Torsten and Claudiu have new powers: they should be able to do some releases of pylint and astroid. To celebrate that and the end of the sprint, we published Pylint 1.3 together with Astroid 1.2. More on this here.

28 Jul 2014 3:21pm GMT

Here are the list of things we managed to achieve during those last two days at EuroPython.

After several attempts, Michal managed to have pylint running analysis on several files in parallel. This is still in a pull request (https://bitbucket.org/logilab/pylint/pull-request/82/added-support-for-checking-files-in) because of some limitations, so we decided it won't be part of the 1.3 release.

Claudiu killed maybe 10 bugs or so and did some heavy issues cleanup in the trackers. He also demonstrated some experimental support of python 3 style annotation to drive a better inference. Pretty exciting! Torsten also killed several bugs, restored python 2.5 compat (though that will need a logilab-common release as well), introduced a new functional test framework that will replace the old one once all the existing tests will be backported. On wednesday, he did show us a near future feature they already have at Google: some kind of confidence level associated to messages so that you can filter out based on that. Sylvain fixed a couple of bugs (including https://bitbucket.org/logilab/pylint/issue/58/ which was annoying all the numpy community), started some refactoring of the PyLinter class so it does a little bit fewer things (still way too many though) and attempted to improve the pylint note on both pylint and astroid, which went down recently "thanks" to the new checks like 'bad-continuation'.

Also, we merged the pylint-brain project into astroid to simplify things, so you should now submit your brain plugins directly to the astroid project. Hopefuly you'll be redirected there on attempt to use the old (removed) pylint-brain project on bitbucket.

And, the good news is that now both Torsten and Claudiu have new powers: they should be able to do some releases of pylint and astroid. To celebrate that and the end of the sprint, we published Pylint 1.3 together with Astroid 1.2. More on this here.

28 Jul 2014 3:21pm GMT

My book, Practical Maya Programming with Python has been finally published! Please check it out and tell me what you think. I hope you will find it sufficiently-but-not-overly opinionated :) It is about as personal as a technical book can get, being distilled from years of mentoring many technical artists and programmers, which is a very intimate experience. It also grows from my belief and observation that becoming a better programmer will, due to all sorts of indirect benefits, help make you a better person.

If you are using Python as a scripting language in a larger application- a game engine, productivity software, 3D software, even a monolithic codebase that no longer feels like Python- there's a lot of relevant material here about turning those environments into more standard and traditional Python development environments, which give you better tools and velocity. The Maya knowledge required is minimal for much of the book. Wrapping a procedural undo system with context managers or decorators is universal. A short quote from the Preface:

This book is not a reference. It is not a cookbook, and it is not a comprehensive guide to Maya's Python API. It is a book that will teach you how to write better Python code for use inside of Maya. It will unearth interesting ways of using Maya and Python to create amazing things that wouldn't be possible otherwise. While there is plenty of code in this book that I encourage you to copy and adapt, this book is not about providing recipes. It is a book to teach skills and enable.

Finally, to those who pre-ordered, I'm truly sorry for all the delays. They're unacceptable. I hope you'll buy and enjoy the book anyway. At least I now have a real-world education on the perils of working with the wrong publisher, and won't be making that same mistake again.

Thanks and happy reading!
Rob Galanakis

28 Jul 2014 2:38pm GMT

My book, Practical Maya Programming with Python has been finally published! Please check it out and tell me what you think. I hope you will find it sufficiently-but-not-overly opinionated :) It is about as personal as a technical book can get, being distilled from years of mentoring many technical artists and programmers, which is a very intimate experience. It also grows from my belief and observation that becoming a better programmer will, due to all sorts of indirect benefits, help make you a better person.

If you are using Python as a scripting language in a larger application- a game engine, productivity software, 3D software, even a monolithic codebase that no longer feels like Python- there's a lot of relevant material here about turning those environments into more standard and traditional Python development environments, which give you better tools and velocity. The Maya knowledge required is minimal for much of the book. Wrapping a procedural undo system with context managers or decorators is universal. A short quote from the Preface:

This book is not a reference. It is not a cookbook, and it is not a comprehensive guide to Maya's Python API. It is a book that will teach you how to write better Python code for use inside of Maya. It will unearth interesting ways of using Maya and Python to create amazing things that wouldn't be possible otherwise. While there is plenty of code in this book that I encourage you to copy and adapt, this book is not about providing recipes. It is a book to teach skills and enable.

Finally, to those who pre-ordered, I'm truly sorry for all the delays. They're unacceptable. I hope you'll buy and enjoy the book anyway. At least I now have a real-world education on the perils of working with the wrong publisher, and won't be making that same mistake again.

Thanks and happy reading!
Rob Galanakis

28 Jul 2014 2:38pm GMT

This year we're trying something new, introductory workshops designed for beginners not attending the conference. It's an attempt by the Python community to reach out to the wider community. All of these workshops are being held at our venue partner, The Edge, part of the State Library of Queensland at South Brisbane.

• DjangoGirls workshop (Monday 4th August, 9am-12pm)
• Introduction to Programming in Python (Friday 1st August, 10am-4pm)
• Introduction to Electronics in Python (Friday 1st August, 10am-4pm)

The full details of the workshops can be found at http://2014.pycon-au.org/programme/workshops

All of our workshops need mentors, please consider signing up as a mentor.

28 Jul 2014 1:41pm GMT

This year we're trying something new, introductory workshops designed for beginners not attending the conference. It's an attempt by the Python community to reach out to the wider community. All of these workshops are being held at our venue partner, The Edge, part of the State Library of Queensland at South Brisbane.

• DjangoGirls workshop (Monday 4th August, 9am-12pm)
• Introduction to Programming in Python (Friday 1st August, 10am-4pm)
• Introduction to Electronics in Python (Friday 1st August, 10am-4pm)

The full details of the workshops can be found at http://2014.pycon-au.org/programme/workshops

All of our workshops need mentors, please consider signing up as a mentor.

28 Jul 2014 1:41pm GMT

I had a fun time at EuroPython 2014 in Berlin last week. It was a very well organized conference and I enjoyed meeting old friends again as well as meeting new people. Before I went I was a bit worried with the amount of attendees it'd feel too massive; I had that experience at a PyCon in the US a few years ago. But I was pleasantly surprised it didn't -- it felt like a smaller conference, and I liked it.

Another positive thing that stood out was a larger diversity; there seemed to be more people from central and eastern Europe there than before, and most of all, there were more women. It was underscored by a 13 year old girl giving a lightning talk -- that was just not going to happen at EuroPython 5 years ago.

This is a very positive trend and I hope it continues. I know it takes a lot of work on the part of the organizers to get this far.

I gave a talk at EuroPython myself this year, and I think it went well:

28 Jul 2014 12:00pm GMT

I had a fun time at EuroPython 2014 in Berlin last week. It was a very well organized conference and I enjoyed meeting old friends again as well as meeting new people. Before I went I was a bit worried with the amount of attendees it'd feel too massive; I had that experience at a PyCon in the US a few years ago. But I was pleasantly surprised it didn't -- it felt like a smaller conference, and I liked it.

Another positive thing that stood out was a larger diversity; there seemed to be more people from central and eastern Europe there than before, and most of all, there were more women. It was underscored by a 13 year old girl giving a lightning talk -- that was just not going to happen at EuroPython 5 years ago.

This is a very positive trend and I hope it continues. I know it takes a lot of work on the part of the organizers to get this far.

I gave a talk at EuroPython myself this year, and I think it went well:

28 Jul 2014 12:00pm GMT

28 Jul 2014 9:04am GMT

28 Jul 2014 9:04am GMT

In this post, we're going to give you a user-friendly introduction to Python decorators. (The code works on both Python 2 [2.6 or 2.7 only] and 3 so don't be concerned with your version.) Before jumping into the topic du jour, consider the usefulness of the map() function. You've got a list with some data and want to apply some function [like times2() below] to all its elements and get a new list with the modified data:

def times2(x):
return x * 2

>>> list(map(times2, [0, 1, 2, 3, 4]))
[0, 2, 4, 6, 8]

Yeah yeah, I know that you can do the same thing with a list comprehension or generator expression, but my point was about an independent piece of logic [like times2()] and mapping that function across a data set ([0, 1, 2, 3, 4]) to generate a new data set ([0, 2, 4, 6, 8]). However, since mapping functions like times2()aren't tied to any particular chunk of data, you can reuse them elsewhere with other unrelated (or related) data.

Along similar lines, consider function calls. You have independent functions and methods in classes. Now, think about "mapped" execution across functions. What are things that you can do with functions that don't have much to do with the behavior of the functions themselves? How about logging function calls, timing them, or some other introspective, cross-cutting behavior. Sure you can implement that behavior in each of the functions that you care about such information, however since they're so generic, it would be nice to only write that logging code just once.

27 Jul 2014 5:09pm GMT

In this post, we're going to give you a user-friendly introduction to Python decorators. (The code works on both Python 2 [2.6 or 2.7 only] and 3 so don't be concerned with your version.) Before jumping into the topic du jour, consider the usefulness of the map() function. You've got a list with some data and want to apply some function [like times2() below] to all its elements and get a new list with the modified data:

def times2(x):
return x * 2

>>> list(map(times2, [0, 1, 2, 3, 4]))
[0, 2, 4, 6, 8]

Yeah yeah, I know that you can do the same thing with a list comprehension or generator expression, but my point was about an independent piece of logic [like times2()] and mapping that function across a data set ([0, 1, 2, 3, 4]) to generate a new data set ([0, 2, 4, 6, 8]). However, since mapping functions like times2()aren't tied to any particular chunk of data, you can reuse them elsewhere with other unrelated (or related) data.

Along similar lines, consider function calls. You have independent functions and methods in classes. Now, think about "mapped" execution across functions. What are things that you can do with functions that don't have much to do with the behavior of the functions themselves? How about logging function calls, timing them, or some other introspective, cross-cutting behavior. Sure you can implement that behavior in each of the functions that you care about such information, however since they're so generic, it would be nice to only write that logging code just once.

27 Jul 2014 5:09pm GMT

Thank you for your suggestion. In our General Assembly at EuroPython 2014 we have just voted on updating the bylaws to also use the "EPS" abbreviation.

We will update the website in the coming days.

Enjoy,
-
EuroPython Society

27 Jul 2014 11:30am GMT

Thank you for your suggestion. In our General Assembly at EuroPython 2014 we have just voted on updating the bylaws to also use the "EPS" abbreviation.

We will update the website in the coming days.

Enjoy,
-
EuroPython Society

27 Jul 2014 11:30am GMT

I am still looking for a good e-mail replacement that is more respectful of my privacy.

This will never happen with the existing e-mail system due to the way it works: when you send an e-mail to someone, even if you encrypt the body of your e-mail, the metadata will transit from server to server in clear, and the final destination will store it.

Every PGP UX I have tried is terrible anyways. It's just too painful to get things right for someone that has no knowledge (and no desire to have some) of how things work.

What I aiming for now is a separate system to send and receive mails with my close friends and my family. Something that my mother can use like regular e-mails, without any extra work.

I guess some kind of "Darknet for E-mails" where they are no intermediate servers between my mailbox and my mom's mailbox, and no way for a eavesdropper to get the content.

Ideally:

• end-to-end encryption
• direct network link between my mom's mail server and me
• based on existing protocols (SMTP/IMAP/POP3) so my mom can use Thunderbird or I can set her up a Zimbra server.

7F9C31FE850E97CEFD4C4591DF93FC757C7C12549DDD55F8EEAECC34FE76C029@tox

26 Jul 2014 11:22am GMT

I am still looking for a good e-mail replacement that is more respectful of my privacy.

This will never happen with the existing e-mail system due to the way it works: when you send an e-mail to someone, even if you encrypt the body of your e-mail, the metadata will transit from server to server in clear, and the final destination will store it.

Every PGP UX I have tried is terrible anyways. It's just too painful to get things right for someone that has no knowledge (and no desire to have some) of how things work.

What I aiming for now is a separate system to send and receive mails with my close friends and my family. Something that my mother can use like regular e-mails, without any extra work.

I guess some kind of "Darknet for E-mails" where they are no intermediate servers between my mailbox and my mom's mailbox, and no way for a eavesdropper to get the content.

Ideally:

• end-to-end encryption
• direct network link between my mom's mail server and me
• based on existing protocols (SMTP/IMAP/POP3) so my mom can use Thunderbird or I can set her up a Zimbra server.

7F9C31FE850E97CEFD4C4591DF93FC757C7C12549DDD55F8EEAECC34FE76C029@tox

26 Jul 2014 11:22am GMT

Zero to hero

What Prompted me??

I started my third year thinking I should do something that would put me different from the rest and one of my professors suggested me as to why don't I apply for GSoC. I don't know why but I took the suggestion rather seriously, thanks to the bet I had with one of my friend(who is about to complete his MBBS) that whoever earns first will buy the other a "RayBan shades". Well, that's it. I was determined. I started my research early, probably during the start of February(I knew I want to buy my friend, his shades and also buy mine too, in the process).

What experiences I had before??

I started looking at previous years' GSoC projects(having had little experience with Open Source) and started learning how to contribute. I was also very fascinated to the amount of knowledge one could gain just by googling and browsing web pages . I discovered very soon, as to what an immensely great tool , email, through which I could chat with anyone in the open source world and ask seemingly stupid questions and always expect to get a gentle reply back with an answer. Well, that held me spell bound and I knew I want to contribute to Open Source.

How did I begin??

About the middle of March, I discovered that my passion for Python as a programming language increased , after understanding how easy it is as a language. Added to that, my popularity among my fellow classmates increased when I started evangelizing Python(thanks to my seniors for introducing it, I guess I did a decent job popularizing the language). And I started contributing to PSF(Python Software Foundation) , started with a simple bug to fix documentation and slowly my interactivity in IRC increased and I started liking one of the project one of the community member proposed.

A twist in the story??

There I was, still a noob and not knowing how to convince my probable mentor that I could complete the project, given direction. About this juncture, a fellow student(from some university in France) mailed this particular mentor that he was interested in the project . Do, remember, I was part of the mailing list and follow the happenings of it. So, I was furious knowing that I had a competition(having put so much effort) and I was not willing to compromise my project (knowing that this is the one project I actually understood and started researching a little bit too). The other projects require me to have some domain knowledge. I went back to my teachers, seniors, friends and Google and started asking the question , "how would i solve the problem the mentor posted?" . I framed a couple of answers, though very noobish , but at least I could reply the email thread posting my understanding of the problem and how I would solve it and also ask various questions I had in my mind. Well, the mentor replied, immediately to my surprise, and responded back with comments as well as answers to the questions I posed. Again, my nemesis/competitor replied back(he having good knowledge about the problem domain). I knew it was not going to be easy. Hence, I went back again, through all my sources, made further understanding of the problem and posted back again. I guess, about 20 mails in the thread , till we(all three of us) decided we should catch up in IRC and discuss more.

The conclusion:

Well, at IRC , most of senior members from the community were present, and they suggested that they should probably extend the scope of the project(since two students were interested in one project and showed immense passion). Unsurprisingly, over multiple meetings, the project scope was expanded, both the students were given equal important but independent tasks and both the students got opportunity to say they are Google Summer of Code students. Thank goodness, we decided to built the project from scratch giving us more than enough work on our plate.

Movie titles:

1) In the open source world, there is no competition , it is only "COLLABORATION".

2) Why give up, when you can win??

3) From Zero to Hero!!

4) A prodigy in making

p.s. I still owe my friend his shades . *sshole, I am still waiting for him to visit me so that I can buy him his shades and buy mine too. Also, I know its been two years since the story happened, but it is never too late to share, don't you agree??

11 Oct 2013 5:39am GMT