23 Oct 2014

feedPlanet Python

Mike Driscoll: PyWin32: How to Get an Application’s Version Number

Occasionally you will need to know what version of software you are using. The normal way to find this information out is usually done by opening the program, going to its Help menu and clicking the About menu item. But this is a Python blog and we want to do it programmatically! To do that on a Windows machine, we need PyWin32. In this article, we'll look at two different methods of getting the version number of an application.


Getting the Version with win32api

First off, we'll get the version number using PyWin32's win32api module. It's actually quite easy to use. Let's take a look:

from win32api import GetFileVersionInfo, LOWORD, HIWORD
 
def get_version_number(filename):
    try:
        info = GetFileVersionInfo (filename, "\\")
        ms = info['FileVersionMS']
        ls = info['FileVersionLS']
        return HIWORD (ms), LOWORD (ms), HIWORD (ls), LOWORD (ls)
    except:
        return "Unknown version"
 
if __name__ == "__main__":
    version = ".".join([str (i) for i in get_version_number (
        r'C:\Program Files\Internet Explorer\iexplore.exe')])
    print version

Here we call GetFileVersionInfo with a path and then attempt to parse the result. If we cannot parse it, then that means that the method didn't return us anything useful and that will cause an exception to be raised. We catch the exception and just return a string that tells us we couldn't find a version number. For this example, we check to see what version of Internet Explorer is installed.


Getting the Version with win32com

To make things more interesting, in the following example we check Google Chrome's version number using PyWin32's win32com module. Let's take a look:

# based on http://stackoverflow.com/questions/580924/python-windows-file-version-attribute
from win32com.client import Dispatch
 
def get_version_via_com(filename):
    parser = Dispatch("Scripting.FileSystemObject")
    version = parser.GetFileVersion(filename)
    return version
 
if __name__ == "__main__":
    path = r"C:\Program Files\Google\Chrome\Application\chrome.exe"
    print get_version_via_com(path)

All we do here is import win32com's Dispatch class and create an instance of that class. Next we call its GetFileVersion method and pass it the path to our executable. Finally we return the result which will be either the number or a message saying that no version information was available. I like this second method a bit more in that it automatically returns a message when no version information was found.


Wrapping Up

Now you know how to check an application version number on Windows. This can be helpful if you need to check if key software needs to be upgraded or perhaps you need to make sure it hasn't been upgraded because some other application requires the older version.


Related Reading

23 Oct 2014 12:30pm GMT

Mike Driscoll: PyWin32: How to Get an Application’s Version Number

Occasionally you will need to know what version of software you are using. The normal way to find this information out is usually done by opening the program, going to its Help menu and clicking the About menu item. But this is a Python blog and we want to do it programmatically! To do that on a Windows machine, we need PyWin32. In this article, we'll look at two different methods of getting the version number of an application.


Getting the Version with win32api

First off, we'll get the version number using PyWin32's win32api module. It's actually quite easy to use. Let's take a look:

from win32api import GetFileVersionInfo, LOWORD, HIWORD
 
def get_version_number(filename):
    try:
        info = GetFileVersionInfo (filename, "\\")
        ms = info['FileVersionMS']
        ls = info['FileVersionLS']
        return HIWORD (ms), LOWORD (ms), HIWORD (ls), LOWORD (ls)
    except:
        return "Unknown version"
 
if __name__ == "__main__":
    version = ".".join([str (i) for i in get_version_number (
        r'C:\Program Files\Internet Explorer\iexplore.exe')])
    print version

Here we call GetFileVersionInfo with a path and then attempt to parse the result. If we cannot parse it, then that means that the method didn't return us anything useful and that will cause an exception to be raised. We catch the exception and just return a string that tells us we couldn't find a version number. For this example, we check to see what version of Internet Explorer is installed.


Getting the Version with win32com

To make things more interesting, in the following example we check Google Chrome's version number using PyWin32's win32com module. Let's take a look:

# based on http://stackoverflow.com/questions/580924/python-windows-file-version-attribute
from win32com.client import Dispatch
 
def get_version_via_com(filename):
    parser = Dispatch("Scripting.FileSystemObject")
    version = parser.GetFileVersion(filename)
    return version
 
if __name__ == "__main__":
    path = r"C:\Program Files\Google\Chrome\Application\chrome.exe"
    print get_version_via_com(path)

All we do here is import win32com's Dispatch class and create an instance of that class. Next we call its GetFileVersion method and pass it the path to our executable. Finally we return the result which will be either the number or a message saying that no version information was available. I like this second method a bit more in that it automatically returns a message when no version information was found.


Wrapping Up

Now you know how to check an application version number on Windows. This can be helpful if you need to check if key software needs to be upgraded or perhaps you need to make sure it hasn't been upgraded because some other application requires the older version.


Related Reading

23 Oct 2014 12:30pm GMT

Lennart Regebro: 59% of maintained packages support Python 3

I ran some statistics on PyPI:

Of the maintained packages:

This means:

So: The ratio of Python 3 support to not Python 3 support is thus 59%, as far as we can tell from the packages themselves.

And if you wonder: "Maintained" means at least one versions released this year (with files uploaded to PyPI) *or* at least 3 versions released the last three years.


Filed under: python, python3 Tagged: python, python 3

23 Oct 2014 10:53am GMT

Lennart Regebro: 59% of maintained packages support Python 3

I ran some statistics on PyPI:

Of the maintained packages:

This means:

So: The ratio of Python 3 support to not Python 3 support is thus 59%, as far as we can tell from the packages themselves.

And if you wonder: "Maintained" means at least one versions released this year (with files uploaded to PyPI) *or* at least 3 versions released the last three years.


Filed under: python, python3 Tagged: python, python 3

23 Oct 2014 10:53am GMT

Kushal Das: More Fedora in life

I am using Fedora from the very fast release. Started contributing to the project from around 2005. I worked on Fedora during my free time, I did that before I joined Red Hat in 2008, during the time I worked in Red Hat and after I left Red Hat last year.

But for the last two weeks I am working on Fedora not only on my free times but also as my day job. I am the Fedora Cloud Engineer as a part of Fedora Engineering team and part of the amazing community of long time Fedora Friends.

23 Oct 2014 9:46am GMT

Kushal Das: More Fedora in life

I am using Fedora from the very fast release. Started contributing to the project from around 2005. I worked on Fedora during my free time, I did that before I joined Red Hat in 2008, during the time I worked in Red Hat and after I left Red Hat last year.

But for the last two weeks I am working on Fedora not only on my free times but also as my day job. I am the Fedora Cloud Engineer as a part of Fedora Engineering team and part of the amazing community of long time Fedora Friends.

23 Oct 2014 9:46am GMT

Kushal Das: Using docker in Fedora for your development work

Last week I worked on DNF for the first time. In this post I am going to explain how I used Docker and a Fedora cloud instance for the same.

I was using a CentOS vm as my primary work system for last two weeks and I had access to a cloud. I created a Fedora 20 instance there.

The first step was to install docker in it and update the system, I also had to upgrade the selinux-policy package and reboot the instance.

# yum upgrade selinux-policy -y; yum update -y
# reboot
# yum install docker-io
# systemctl start docker
# systemctl enable docker

Then pull in the Fedora 21 Docker image.

# docker pull fedora:21

The above command will take time as it will download the image. After this we will start a Fedora 21 container.

# docker run -t -i fedora:21 /bin/bash

We will install all the required dependencies in the image, use yum as you do normally and then get out by pressing Crrl+d.

[root@3e5de622ac00 /]# yum install dnf python-nose python-mock cmake -y

Now we can commit this as a new image so that we can reuse it in the future. We do this by docker commit command.

#  docker commit -m "with dnf" -a "Kushal Das" 3e5de622ac00 kushaldas/dnfimage

After this the only thing left to start a container with this newly created image and mounted directory from host machine.

# docker run -t -i -v /opt/dnf:/opt/dnf kushaldas/dnfimage /bin/bash

This command assumes the code is already in the /opt/dnf of the host system. Even if I managed to do something bad in that container, my actual host is safe. I just have to get out of the container and start a new one.

23 Oct 2014 9:30am GMT

Kushal Das: Using docker in Fedora for your development work

Last week I worked on DNF for the first time. In this post I am going to explain how I used Docker and a Fedora cloud instance for the same.

I was using a CentOS vm as my primary work system for last two weeks and I had access to a cloud. I created a Fedora 20 instance there.

The first step was to install docker in it and update the system, I also had to upgrade the selinux-policy package and reboot the instance.

# yum upgrade selinux-policy -y; yum update -y
# reboot
# yum install docker-io
# systemctl start docker
# systemctl enable docker

Then pull in the Fedora 21 Docker image.

# docker pull fedora:21

The above command will take time as it will download the image. After this we will start a Fedora 21 container.

# docker run -t -i fedora:21 /bin/bash

We will install all the required dependencies in the image, use yum as you do normally and then get out by pressing Crrl+d.

[root@3e5de622ac00 /]# yum install dnf python-nose python-mock cmake -y

Now we can commit this as a new image so that we can reuse it in the future. We do this by docker commit command.

#  docker commit -m "with dnf" -a "Kushal Das" 3e5de622ac00 kushaldas/dnfimage

After this the only thing left to start a container with this newly created image and mounted directory from host machine.

# docker run -t -i -v /opt/dnf:/opt/dnf kushaldas/dnfimage /bin/bash

This command assumes the code is already in the /opt/dnf of the host system. Even if I managed to do something bad in that container, my actual host is safe. I just have to get out of the container and start a new one.

23 Oct 2014 9:30am GMT

22 Oct 2014

feedPlanet Python

Obey the Testing Goat: Decorators!

Someone recently wrote to me asking about decorators, and saying they found them a bit confusing. Here's a post based on the email I replied to them with.

The best way to understand decorators is to build a couple of them, so here are two examples for you to try out. The first is in the Django world, the second is actually a simpler, pure-python one.

Challenge: build a decorator in a simple Django app

We've built a very basic todo lists app using Django. It has views to deal with viewing lists, creating new lists, and adding to existing lists. Two of these views end up doing some similar work, which is to retrieve a list object from the database based on its list ID:

def add_item(request, list_id):
    list_ = List.objects.get(id=list_id)
    Item.objects.create(text=request.POST['item_text'], list=list_)
    return redirect('/lists/%d/' % (list_.id,))


def view_list(request, list_id):
    list_ = List.objects.get(id=list_id)
    return render(request, 'list.html', {'list': list_})

(Full code here)

This is a good use case for a decorator.

A decorator can be used to extract duplicated work, and also to change the arguments to a function. So we should be able to build a decorator that does the list-getting for us. Here's the target:

@get_list
def add_item(request, list_):
    Item.objects.create(text=request.POST['item_text'], list=list_)
    return redirect('/lists/%d/' % (list_.id,))


@get_list
def view_list(request, list_):
    return render(request, 'list.html', {'list': list_})

So how do we build a decorator that does that? A decorator is a function that takes a function, and returns another function that does a slightly modified version of the work the original function was doing. We want our decorator to transform the simplified view functions we have above, into something that looks like the original functions.

(you end up saying "function" a lot in any explanation of decorators...)

Here's a template:

def get_list(view_fn):

    def decorated_view(...?):
        ???
        return view_fn(...?)

    return decorated_view

Can you get it working? Thankfully, our code has tests, so they'll tell you when you get it right...

git clone -b chapter_06 https://github.com/hjwp/book-example
python3 manage.py test lists # dependencies: django 1.7

Some rules of thumb for decorators:

Decorators definitely are a bit brain-melting, so it may take a bit of effort to wrap your head around it. Once you get the hang of them, they're dead useful though,

A simpler decorator challenge:

If you're finding it impossible, you could start with a simpler challenge... say, building a decorator to make functions return an absolute value:

def absolute(fn):
    # this decorator currently does nothing
    def modified_fn(x):
        return fn(x)
    return modified_fn


def foo(x):
    return 1 - x

assert foo(3) == -2


@absolute
def foo(x):
    return 1 - x

assert foo(3) == 2  # this will fail, get is passing!
git clone https://gist.github.com/2cc523b66d9c0fe41c4b.git deccy
python3 deccy/deccy.py

Enjoy!

22 Oct 2014 11:00pm GMT

Obey the Testing Goat: Decorators!

Someone recently wrote to me asking about decorators, and saying they found them a bit confusing. Here's a post based on the email I replied to them with.

The best way to understand decorators is to build a couple of them, so here are two examples for you to try out. The first is in the Django world, the second is actually a simpler, pure-python one.

Challenge: build a decorator in a simple Django app

We've built a very basic todo lists app using Django. It has views to deal with viewing lists, creating new lists, and adding to existing lists. Two of these views end up doing some similar work, which is to retrieve a list object from the database based on its list ID:

def add_item(request, list_id):
    list_ = List.objects.get(id=list_id)
    Item.objects.create(text=request.POST['item_text'], list=list_)
    return redirect('/lists/%d/' % (list_.id,))


def view_list(request, list_id):
    list_ = List.objects.get(id=list_id)
    return render(request, 'list.html', {'list': list_})

(Full code here)

This is a good use case for a decorator.

A decorator can be used to extract duplicated work, and also to change the arguments to a function. So we should be able to build a decorator that does the list-getting for us. Here's the target:

@get_list
def add_item(request, list_):
    Item.objects.create(text=request.POST['item_text'], list=list_)
    return redirect('/lists/%d/' % (list_.id,))


@get_list
def view_list(request, list_):
    return render(request, 'list.html', {'list': list_})

So how do we build a decorator that does that? A decorator is a function that takes a function, and returns another function that does a slightly modified version of the work the original function was doing. We want our decorator to transform the simplified view functions we have above, into something that looks like the original functions.

(you end up saying "function" a lot in any explanation of decorators...)

Here's a template:

def get_list(view_fn):

    def decorated_view(...?):
        ???
        return view_fn(...?)

    return decorated_view

Can you get it working? Thankfully, our code has tests, so they'll tell you when you get it right...

git clone -b chapter_06 https://github.com/hjwp/book-example
python3 manage.py test lists # dependencies: django 1.7

Some rules of thumb for decorators:

Decorators definitely are a bit brain-melting, so it may take a bit of effort to wrap your head around it. Once you get the hang of them, they're dead useful though,

A simpler decorator challenge:

If you're finding it impossible, you could start with a simpler challenge... say, building a decorator to make functions return an absolute value:

def absolute(fn):
    # this decorator currently does nothing
    def modified_fn(x):
        return fn(x)
    return modified_fn


def foo(x):
    return 1 - x

assert foo(3) == -2


@absolute
def foo(x):
    return 1 - x

assert foo(3) == 2  # this will fail, get is passing!
git clone https://gist.github.com/2cc523b66d9c0fe41c4b.git deccy
python3 deccy/deccy.py

Enjoy!

22 Oct 2014 11:00pm GMT

François Dion: A Tale of 2 talks

Am I in over my head? tackling two different python talks this coming monday @WakeForest1834 #wsnc 1 on MMS, flask and 2 on #python 2 to 3
- Francois Dion (@f_dion) October 23, 2014


http://www.meetup.com/PYthon-Piedmont-Triad-User-Group-PYPTUG/events/213427092/

22 Oct 2014 10:45pm GMT

François Dion: A Tale of 2 talks

Am I in over my head? tackling two different python talks this coming monday @WakeForest1834 #wsnc 1 on MMS, flask and 2 on #python 2 to 3
- Francois Dion (@f_dion) October 23, 2014


http://www.meetup.com/PYthon-Piedmont-Triad-User-Group-PYPTUG/events/213427092/

22 Oct 2014 10:45pm GMT

Python Piedmont Triad User Group: PYPTUG Meeting - October 27th

PYthon Piedmont Triad User Group meeting

Come join PYPTUG at out next meeting (September 29th 2014) to learn more about the Python programming language, modules and tools. Python is the perfect language to learn if you've never programmed before, and at the other end, it is also the perfect tool that no expert would do without.


What

Meeting will start at 5:30pm.

We will open on an Intro to PYPTUG and on how to get started with Python, PYPTUG activities and members projects, then on to News from the community.

This month we will have a tutorial review followed by a main talk.

Internet Tutorial Review

Continuing on the review last month of Gizeh (Cairo for Tourists), this month we will review an Internet tutorial on creating digital coupons: "Branded MMS coupon generation", and a few ways that this could be made better. This should be of interest to many: mobile devs, devops, infrastructure architects, web app devs, marketers, CIO/CTOs.

Main Talk

by Francois Dion
Title: "Mystery Python Theater 3K: What should be your next step"

Bio: Francois Dion is the founder of PYPTUG. In the few words of his blog's profile he is an "Entrepreneur, Hacker, Mentor, Polyglot, Polymath, Musician, Photographer"

Abstract: Francois will talk about Python 3, why it should be on your radar, what are some differences, how you should prepare for a transition. He will also review some specific cases in different fields, and what kind of changes had to be done, such as the case of the MMA software, a "Band in a box" style python program that let's you create full midi scores from basic chords, with Python 2 or 3.

Lightning talks!


We will have some time for extemporaneous "lightning talks" of 5-10 minute duration. If you'd like to do one, some suggestions of talks were provided here, if you are looking for inspiration. Or talk about a project you are working on.






When

Monday, October 27th 2014
Meeting starts at 5:30PM

Where

Wake Forest University,

close to Polo Rd and University Parkway:

Manchester Hall
room: Manchester 241
Wake Forest University, Winston-Salem, NC 27109

Map this

See also this campus map (PDF) and also the Parking Map (PDF) (Manchester hall is #20A on the parking map)

And speaking of parking: Parking after 5pm is on a first-come, first-serve basis. The official parking policy is:
"Visitors can park in any general parking lot on campus. Visitors should avoid reserved spaces, faculty/staff lots, fire lanes or other restricted area on campus. Frequent visitors should contact Parking and Transportation to register for a parking permit."

Mailing List


Don't forget to sign up to our user group mailing list:

https://groups.google.com/d/forum/pyptug?hl=en

It is the only step required to become a PYPTUG member.

Meetup Group


In order to get a feel for how much food we'll need, we ask that you register your attendance to this meeting on meetup:

http://www.meetup.com/PYthon-Piedmont-Triad-User-Group-PYPTUG/events/213427092/

22 Oct 2014 7:53pm GMT

Python Piedmont Triad User Group: PYPTUG Meeting - October 27th

PYthon Piedmont Triad User Group meeting

Come join PYPTUG at out next meeting (September 29th 2014) to learn more about the Python programming language, modules and tools. Python is the perfect language to learn if you've never programmed before, and at the other end, it is also the perfect tool that no expert would do without.


What

Meeting will start at 5:30pm.

We will open on an Intro to PYPTUG and on how to get started with Python, PYPTUG activities and members projects, then on to News from the community.

This month we will have a tutorial review followed by a main talk.

Internet Tutorial Review

Continuing on the review last month of Gizeh (Cairo for Tourists), this month we will review an Internet tutorial on creating digital coupons: "Branded MMS coupon generation", and a few ways that this could be made better. This should be of interest to many: mobile devs, devops, infrastructure architects, web app devs, marketers, CIO/CTOs.

Main Talk

by Francois Dion
Title: "Mystery Python Theater 3K: What should be your next step"

Bio: Francois Dion is the founder of PYPTUG. In the few words of his blog's profile he is an "Entrepreneur, Hacker, Mentor, Polyglot, Polymath, Musician, Photographer"

Abstract: Francois will talk about Python 3, why it should be on your radar, what are some differences, how you should prepare for a transition. He will also review some specific cases in different fields, and what kind of changes had to be done, such as the case of the MMA software, a "Band in a box" style python program that let's you create full midi scores from basic chords, with Python 2 or 3.

Lightning talks!


We will have some time for extemporaneous "lightning talks" of 5-10 minute duration. If you'd like to do one, some suggestions of talks were provided here, if you are looking for inspiration. Or talk about a project you are working on.






When

Monday, October 27th 2014
Meeting starts at 5:30PM

Where

Wake Forest University,

close to Polo Rd and University Parkway:

Manchester Hall
room: Manchester 241
Wake Forest University, Winston-Salem, NC 27109

Map this

See also this campus map (PDF) and also the Parking Map (PDF) (Manchester hall is #20A on the parking map)

And speaking of parking: Parking after 5pm is on a first-come, first-serve basis. The official parking policy is:
"Visitors can park in any general parking lot on campus. Visitors should avoid reserved spaces, faculty/staff lots, fire lanes or other restricted area on campus. Frequent visitors should contact Parking and Transportation to register for a parking permit."

Mailing List


Don't forget to sign up to our user group mailing list:

https://groups.google.com/d/forum/pyptug?hl=en

It is the only step required to become a PYPTUG member.

Meetup Group


In order to get a feel for how much food we'll need, we ask that you register your attendance to this meeting on meetup:

http://www.meetup.com/PYthon-Piedmont-Triad-User-Group-PYPTUG/events/213427092/

22 Oct 2014 7:53pm GMT

Mike Driscoll: PyWin32: How to Set the Desktop Background on Windows

Back in my system administrator days, we were thinking about setting the user's Window desktop background to a specific image on login. Since I was in charge of the login scripts, which were written in Python, I decided to do some research to find out if there was a way to do it. We will look at two different approaches to this task in this article. The code in this article was tested using Python 2.7.8 and PyWin32 219 on Windows 7.

For the first approach, you will need to go download a copy of PyWin32 and install it. Now let's look at the code:

# based on http://dzone.com/snippets/set-windows-desktop-wallpaper
import win32api, win32con, win32gui
 
#----------------------------------------------------------------------
def setWallpaper(path):
    key = win32api.RegOpenKeyEx(win32con.HKEY_CURRENT_USER,"Control Panel\\Desktop",0,win32con.KEY_SET_VALUE)
    win32api.RegSetValueEx(key, "WallpaperStyle", 0, win32con.REG_SZ, "0")
    win32api.RegSetValueEx(key, "TileWallpaper", 0, win32con.REG_SZ, "0")
    win32gui.SystemParametersInfo(win32con.SPI_SETDESKWALLPAPER, path, 1+2)
 
if __name__ == "__main__":
    path = r'C:\Users\Public\Pictures\Sample Pictures\Jellyfish.jpg'
    setWallpaper(path)

In this example, we use a sample image that Microsoft provides with Windows. In the code above, we eidt a Windows Registry key. You could do the first 3 lines using Python's own _winreg module if you wanted to. The last line tells Windows to set the desktop to the image we provided.

Now let's look at another approach that utilizes the ctypes module and PyWin32.

import ctypes
import win32con
 
def setWallpaperWithCtypes(path):
    # This code is based on the following two links
    # http://mail.python.org/pipermail/python-win32/2005-January/002893.html
    # http://code.activestate.com/recipes/435877-change-the-wallpaper-under-windows/
    cs = ctypes.c_buffer(path)
    ok = ctypes.windll.user32.SystemParametersInfoA(win32con.SPI_SETDESKWALLPAPER, 0, cs, 0)
 
if __name__ == "__main__":
    path = r'C:\Users\Public\Pictures\Sample Pictures\Jellyfish.jpg'
    setWallpaperWithCtypes(path)

In this piece of code, we create a buffer object that we then pass to basically the same command as we did in the previous example, namely SystemParametersInfoA. You will note that we don't need to edit the registry in this latter case. If you check out the links listed in the sample code, you will note that some users found that Windows XP only allowed bitmaps to be set as the desktop background. I tested with a JPEG on Windows 7 and it worked fine for me.

Now you can create your own script that changes the desktop's background at random intervals! Or maybe you'll just use this knowledge in your own system administration duties. Have fun!

22 Oct 2014 5:15pm GMT

Mike Driscoll: PyWin32: How to Set the Desktop Background on Windows

Back in my system administrator days, we were thinking about setting the user's Window desktop background to a specific image on login. Since I was in charge of the login scripts, which were written in Python, I decided to do some research to find out if there was a way to do it. We will look at two different approaches to this task in this article. The code in this article was tested using Python 2.7.8 and PyWin32 219 on Windows 7.

For the first approach, you will need to go download a copy of PyWin32 and install it. Now let's look at the code:

# based on http://dzone.com/snippets/set-windows-desktop-wallpaper
import win32api, win32con, win32gui
 
#----------------------------------------------------------------------
def setWallpaper(path):
    key = win32api.RegOpenKeyEx(win32con.HKEY_CURRENT_USER,"Control Panel\\Desktop",0,win32con.KEY_SET_VALUE)
    win32api.RegSetValueEx(key, "WallpaperStyle", 0, win32con.REG_SZ, "0")
    win32api.RegSetValueEx(key, "TileWallpaper", 0, win32con.REG_SZ, "0")
    win32gui.SystemParametersInfo(win32con.SPI_SETDESKWALLPAPER, path, 1+2)
 
if __name__ == "__main__":
    path = r'C:\Users\Public\Pictures\Sample Pictures\Jellyfish.jpg'
    setWallpaper(path)

In this example, we use a sample image that Microsoft provides with Windows. In the code above, we eidt a Windows Registry key. You could do the first 3 lines using Python's own _winreg module if you wanted to. The last line tells Windows to set the desktop to the image we provided.

Now let's look at another approach that utilizes the ctypes module and PyWin32.

import ctypes
import win32con
 
def setWallpaperWithCtypes(path):
    # This code is based on the following two links
    # http://mail.python.org/pipermail/python-win32/2005-January/002893.html
    # http://code.activestate.com/recipes/435877-change-the-wallpaper-under-windows/
    cs = ctypes.c_buffer(path)
    ok = ctypes.windll.user32.SystemParametersInfoA(win32con.SPI_SETDESKWALLPAPER, 0, cs, 0)
 
if __name__ == "__main__":
    path = r'C:\Users\Public\Pictures\Sample Pictures\Jellyfish.jpg'
    setWallpaperWithCtypes(path)

In this piece of code, we create a buffer object that we then pass to basically the same command as we did in the previous example, namely SystemParametersInfoA. You will note that we don't need to edit the registry in this latter case. If you check out the links listed in the sample code, you will note that some users found that Windows XP only allowed bitmaps to be set as the desktop background. I tested with a JPEG on Windows 7 and it worked fine for me.

Now you can create your own script that changes the desktop's background at random intervals! Or maybe you'll just use this knowledge in your own system administration duties. Have fun!

22 Oct 2014 5:15pm GMT

PyPy Development: PyPy3 2.4.0 released

We're pleased to announce the availability of PyPy3 2.4.0!

This release contains several bugfixes and enhancements. Among the user-facing improvements specific to PyPy3:
  • Better Windows compatibility, e.g. the nt module functions _getfinalpathname & _getfileinformation are now supported (the former is required for the popular pathlib library for example)
  • Various fsencode PEP 383 related fixes to the posix module (readlink, uname, ttyname and ctermid) and improved locale handling
  • Switched the default binary name on POSIX distributions from 'pypy' to 'pypy3' (which symlinks to to 'pypy3.2')
  • Fixed a couple different crashes related to parsing Python 3 source code

And improvements shared with the recent PyPy 2.4.0 release:
  • internal refactoring in string and GIL handling which led to significant speedups
  • improved handling of multiple objects (like sockets) in long-running programs. They are collected and released more efficiently, reducing memory use. In simpler terms - we closed what looked like a memory leak
  • Windows builds now link statically to zlib, expat, bzip, and openssl-1.0.1i
  • Many issues were resolved since the 2.3.1 release in June

You can download PyPy3 2.4.0 here http://pypy.org/download.html.

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 and 3.2.5. It's fast (pypy 2.4 and cpython 2.7.x performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows, and OpenBSD, as well as newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux.
We would like to thank our donors for the continued support of the PyPy project.

The complete release notice is here.

Please try it out and let us know what you think. We especially welcome success stories, please tell us about how it has helped you!

Cheers, The PyPy Team

22 Oct 2014 4:41pm GMT

PyPy Development: PyPy3 2.4.0 released

We're pleased to announce the availability of PyPy3 2.4.0!

This release contains several bugfixes and enhancements. Among the user-facing improvements specific to PyPy3:
  • Better Windows compatibility, e.g. the nt module functions _getfinalpathname & _getfileinformation are now supported (the former is required for the popular pathlib library for example)
  • Various fsencode PEP 383 related fixes to the posix module (readlink, uname, ttyname and ctermid) and improved locale handling
  • Switched the default binary name on POSIX distributions from 'pypy' to 'pypy3' (which symlinks to to 'pypy3.2')
  • Fixed a couple different crashes related to parsing Python 3 source code

And improvements shared with the recent PyPy 2.4.0 release:
  • internal refactoring in string and GIL handling which led to significant speedups
  • improved handling of multiple objects (like sockets) in long-running programs. They are collected and released more efficiently, reducing memory use. In simpler terms - we closed what looked like a memory leak
  • Windows builds now link statically to zlib, expat, bzip, and openssl-1.0.1i
  • Many issues were resolved since the 2.3.1 release in June

You can download PyPy3 2.4.0 here http://pypy.org/download.html.

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 and 3.2.5. It's fast (pypy 2.4 and cpython 2.7.x performance comparison) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows, and OpenBSD, as well as newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux.
We would like to thank our donors for the continued support of the PyPy project.

The complete release notice is here.

Please try it out and let us know what you think. We especially welcome success stories, please tell us about how it has helped you!

Cheers, The PyPy Team

22 Oct 2014 4:41pm GMT

Calvin Spealman: Dead To Me! DomNomNom

DomNomNom was a toy templating tool for front-end web applications I built during a long ride in the passenger seat. The idea was to build a templating system that required minimal, and in many cases, no template at all. I wanted to see if it was possible to map data directly into markup structures based purely on semantics.


For example, instead of some mark up that rendered a title into the page like {{ title }} we might just map the <h1> tag to the title in the data binding.


$("body").domnomnom({
"h1": "This is the title",
})


And it was really easy to get this basic setup in place quickly. I began to take it further. I allowed mapping lists of data, which would clone an element instead of simply inserting the text contents into it. Suddenly I could render tables and lists with ease.


"ul": {
"li": ["one", "two", "three"]
}


And the markup's original <li> would function as a template to clone for this content. It was very clean to write templates for, because they were just mark-up with dummy data and content in them. This meant a designer could build the templates with whatever tools they wanted and the data could just get pumped into it.


DomNomNom in its final state supports mapping syntax that can handle attributes and properties, so you can map into form fields and the like. There are also controls capable of changing the order cloned elements are inserted and allowing the clone templates to be controlled better. If I removed the empty lines for formatting, the whole thing would come in under 100 lines of Javascript.


I built this on jQuery, but if I re-did this based on modern browsers with querySelector it probably wouldn't grow by more than a dozen lines, and would be a lot faster.


Check it out, if just to see.

22 Oct 2014 2:30pm GMT

Calvin Spealman: Dead To Me! DomNomNom

DomNomNom was a toy templating tool for front-end web applications I built during a long ride in the passenger seat. The idea was to build a templating system that required minimal, and in many cases, no template at all. I wanted to see if it was possible to map data directly into markup structures based purely on semantics.


For example, instead of some mark up that rendered a title into the page like {{ title }} we might just map the <h1> tag to the title in the data binding.


$("body").domnomnom({
"h1": "This is the title",
})


And it was really easy to get this basic setup in place quickly. I began to take it further. I allowed mapping lists of data, which would clone an element instead of simply inserting the text contents into it. Suddenly I could render tables and lists with ease.


"ul": {
"li": ["one", "two", "three"]
}


And the markup's original <li> would function as a template to clone for this content. It was very clean to write templates for, because they were just mark-up with dummy data and content in them. This meant a designer could build the templates with whatever tools they wanted and the data could just get pumped into it.


DomNomNom in its final state supports mapping syntax that can handle attributes and properties, so you can map into form fields and the like. There are also controls capable of changing the order cloned elements are inserted and allowing the clone templates to be controlled better. If I removed the empty lines for formatting, the whole thing would come in under 100 lines of Javascript.


I built this on jQuery, but if I re-did this based on modern browsers with querySelector it probably wouldn't grow by more than a dozen lines, and would be a lot faster.


Check it out, if just to see.

22 Oct 2014 2:30pm GMT

Django Weblog: Bugfix releases issued

Today we're issuing the first bugfix release in the 1.7 release series, along with bugfix releases for each of the other supported release series (1.4 and 1.6), plus the final bugfix release of the 1.5 series. Although Django 1.5 is technically already past its end-of-life, a regression discovered in the last 1.5 release prompted the core team to decide to issue one final release for users of Django 1.5.

Details can be found in the release notes for each release series:

The release packages and checksums are, as always, available from our downloads page, as well as from the Python Package Index.

22 Oct 2014 11:00am GMT

Django Weblog: Bugfix releases issued

Today we're issuing the first bugfix release in the 1.7 release series, along with bugfix releases for each of the other supported release series (1.4 and 1.6), plus the final bugfix release of the 1.5 series. Although Django 1.5 is technically already past its end-of-life, a regression discovered in the last 1.5 release prompted the core team to decide to issue one final release for users of Django 1.5.

Details can be found in the release notes for each release series:

The release packages and checksums are, as always, available from our downloads page, as well as from the Python Package Index.

22 Oct 2014 11:00am GMT

Denis Kurov: Enlivepy A Different Approach to Html Templating in Python

Don't worry - this blog entry won't consist of me rambling about yet another templating engine that will change the world. Instead, I'll be focusing on a practical aspect - the problems the current HTML templating libraries have and what can be possibly done about these issues.
Django and all other MVC/MVP frameworks do a great job at separating the Controller logic from the View logic. This way you can have all your complex operations in Controller and send back the data that needs to be shown or rendered to the end user. On the other end of this process you have a templating library - it takes Python objects and renders them.
Now, some of these templating libraries argue that having lots of logic in the representation layer is a bad thing (Django templating for example). Others, like Jinja2, are more lax and allow you to do lots of interesting things.
In both cases what we are actually doing is mixing data with logic. And while this might not seem like a problem with small applications, when you start operating on a greater scale, issues start to appear (a lot). Let me provide a few examples:
Read the rest of the article here.

22 Oct 2014 5:17am GMT

Denis Kurov: Enlivepy A Different Approach to Html Templating in Python

Don't worry - this blog entry won't consist of me rambling about yet another templating engine that will change the world. Instead, I'll be focusing on a practical aspect - the problems the current HTML templating libraries have and what can be possibly done about these issues.
Django and all other MVC/MVP frameworks do a great job at separating the Controller logic from the View logic. This way you can have all your complex operations in Controller and send back the data that needs to be shown or rendered to the end user. On the other end of this process you have a templating library - it takes Python objects and renders them.
Now, some of these templating libraries argue that having lots of logic in the representation layer is a bad thing (Django templating for example). Others, like Jinja2, are more lax and allow you to do lots of interesting things.
In both cases what we are actually doing is mixing data with logic. And while this might not seem like a problem with small applications, when you start operating on a greater scale, issues start to appear (a lot). Let me provide a few examples:
Read the rest of the article here.

22 Oct 2014 5:17am GMT

21 Oct 2014

feedPlanet Python

Mike Driscoll: Logging Currently Running Processes with Python

I was looking through some of my old code and noticed this old script where I was creating a log of all running processes every 5 minutes. I believe I originally wrote the code to help me diagnose rogue processes that were eating memory or pegging the CPU. I was using the psutil project to get the information I needed, so if you'd like to follow along you will need to download and install it as well.

Here's the code:

import os
import psutil
import time
 
#----------------------------------------------------------------------
def create_process_logs(log_dir):
    """
    Create a log of all the currently running processes
    """
    if not os.path.exists(log_dir):
        try:
            os.mkdir(log_dir)
        except:
            pass
 
 
    separator = "-" * 80
    col_format = "%7s %7s %12s %12s %30s"
    data_format = "%7.4f %7.2f %12s %12s %30s"
    while 1:
        procs = psutil.get_process_list()
        procs = sorted(procs, key=lambda proc: proc.name)
 
        log_path = os.path.join(log_dir, "procLog%i.log" % int(time.time()))
        f = open(log_path, 'w')
        f.write(separator + "\n")
        f.write(time.ctime() + "\n")
        f.write(col_format % ("%CPU", "%MEM", "VMS", "RSS", "NAME"))
        f.write("\n")
 
        for proc in procs:
            cpu_percent = proc.get_cpu_percent()
            mem_percent = proc.get_memory_percent()
            rss, vms = proc.get_memory_info()
            rss = str(rss)
            vms = str(vms)
            name = proc.name
            f.write(data_format % (cpu_percent, mem_percent, vms, rss, name))
            f.write("\n\n")
        f.close()
        print "Finished log update!"
        time.sleep(300)
        print "writing new log data!"
 
if __name__ == "__main__":
    log_dir = r"c:\users\USERNAME\documents"
    create_process_logs(log_dir)

Let's break this down a bit. Here we pass in a log directory, check if it exists and create it if it does not. Next we set up a few variables that contain formatting for the log file. Then we start an infinite loop that uses psutil to get all the currently running processes. We also sort the processes by name. Next, we open up a uniquely named log file and we write out each process'es CPU and memory usage along with it's VMS, RSS and name of the executable. Then we close the file and wait 5 minutes before doing it all over again.

In retrospect, it would probably have been better to write this information to a database like SQLite so that the data could be searched and graphed. In the meantime, hopefully you will find some useful tidbits in here that you can use for your own project.

21 Oct 2014 10:15pm GMT

Mike Driscoll: Logging Currently Running Processes with Python

I was looking through some of my old code and noticed this old script where I was creating a log of all running processes every 5 minutes. I believe I originally wrote the code to help me diagnose rogue processes that were eating memory or pegging the CPU. I was using the psutil project to get the information I needed, so if you'd like to follow along you will need to download and install it as well.

Here's the code:

import os
import psutil
import time
 
#----------------------------------------------------------------------
def create_process_logs(log_dir):
    """
    Create a log of all the currently running processes
    """
    if not os.path.exists(log_dir):
        try:
            os.mkdir(log_dir)
        except:
            pass
 
 
    separator = "-" * 80
    col_format = "%7s %7s %12s %12s %30s"
    data_format = "%7.4f %7.2f %12s %12s %30s"
    while 1:
        procs = psutil.get_process_list()
        procs = sorted(procs, key=lambda proc: proc.name)
 
        log_path = os.path.join(log_dir, "procLog%i.log" % int(time.time()))
        f = open(log_path, 'w')
        f.write(separator + "\n")
        f.write(time.ctime() + "\n")
        f.write(col_format % ("%CPU", "%MEM", "VMS", "RSS", "NAME"))
        f.write("\n")
 
        for proc in procs:
            cpu_percent = proc.get_cpu_percent()
            mem_percent = proc.get_memory_percent()
            rss, vms = proc.get_memory_info()
            rss = str(rss)
            vms = str(vms)
            name = proc.name
            f.write(data_format % (cpu_percent, mem_percent, vms, rss, name))
            f.write("\n\n")
        f.close()
        print "Finished log update!"
        time.sleep(300)
        print "writing new log data!"
 
if __name__ == "__main__":
    log_dir = r"c:\users\USERNAME\documents"
    create_process_logs(log_dir)

Let's break this down a bit. Here we pass in a log directory, check if it exists and create it if it does not. Next we set up a few variables that contain formatting for the log file. Then we start an infinite loop that uses psutil to get all the currently running processes. We also sort the processes by name. Next, we open up a uniquely named log file and we write out each process'es CPU and memory usage along with it's VMS, RSS and name of the executable. Then we close the file and wait 5 minutes before doing it all over again.

In retrospect, it would probably have been better to write this information to a database like SQLite so that the data could be searched and graphed. In the meantime, hopefully you will find some useful tidbits in here that you can use for your own project.

21 Oct 2014 10:15pm GMT

20 Oct 2014

feedPlanet Python

Mike Driscoll: PyWin32 – How to Bring a Window to Front

I recently saw someone asking how to bring a window to the front in Windows and I realized I had had some old unreleased code that might help someone with this task. A long time ago, Tim Golden (and possibly some other fellows on the PyWin32 mailing list) showed me how to make windows come to the front on Windows XP, although it should be noted that it also works on Windows 7. If you'd like to follow along, you will need to download and install your own copy of PyWin32.

We will need to choose something to bring to the front. I like to use Notepad for testing as I know it will be on every Windows desktop in existence. Open up Notepad and then put some other application's window in front of it.

Now we're ready to look at some code:

import win32gui
 
def windowEnumerationHandler(hwnd, top_windows):
    top_windows.append((hwnd, win32gui.GetWindowText(hwnd)))
 
if __name__ == "__main__":
    results = []
    top_windows = []
    win32gui.EnumWindows(windowEnumerationHandler, top_windows)
    for i in top_windows:
        if "notepad" in i[1].lower():
            print i
            win32gui.ShowWindow(i[0],5)
            win32gui.SetForegroundWindow(i[0])
            break

We only need PyWin32's win32gui module for this little script. We write a little function that takes a window handle and a Python list. Then we call win32gui's EnumWindows method, which takes a callback and an extra argument that is a Python object. According to the documentation, the EnumWindows method "Enumerates all top-level windows on the screen by passing the handle to each window, in turn, to an application-defined callback function". So we pass it our method and it enumerates the windows, passing a handle of each window plus our Python list to our function. It works kind of like a messed up decorator.

Once that's done, your top_windows list will be full of lots of items, most of which you didn't even know were running. You can print that our and inspect your results if you like. It's really quite intereting. But for our purposes, we will skip that and just loop over the list, looking for the word "Notepad". Once we find it, we use win32gui's ShowWindow and SetForegroundWindow methods to bring the application to the foreground.

Note that really need to look for a unique string so that you bring up the right window. What would happen if you had multiple Notepad instance running with different files open? With the current code, you would bring the first Notepad instance that it found forward, which might not be what you want.

You may be wondering why anyone would even want to go to the trouble of doing this in the first place. In my case, I once had a project where I had to bring a certain window to the foreground and enter automate it using SendKeys. It was an ugly piece of brittle code that I wouldn't wish on anyone. Fortunately, there are better tools for that sort of thing nowadays such as pywinauto, but you still might find this code helpful in something esoteric that is thrown your way. Have fun!

Note: This code was tested using Python 2.7.8 and PyWin32 219 on Windows 7.

20 Oct 2014 10:15pm GMT

Mike Driscoll: PyWin32 – How to Bring a Window to Front

I recently saw someone asking how to bring a window to the front in Windows and I realized I had had some old unreleased code that might help someone with this task. A long time ago, Tim Golden (and possibly some other fellows on the PyWin32 mailing list) showed me how to make windows come to the front on Windows XP, although it should be noted that it also works on Windows 7. If you'd like to follow along, you will need to download and install your own copy of PyWin32.

We will need to choose something to bring to the front. I like to use Notepad for testing as I know it will be on every Windows desktop in existence. Open up Notepad and then put some other application's window in front of it.

Now we're ready to look at some code:

import win32gui
 
def windowEnumerationHandler(hwnd, top_windows):
    top_windows.append((hwnd, win32gui.GetWindowText(hwnd)))
 
if __name__ == "__main__":
    results = []
    top_windows = []
    win32gui.EnumWindows(windowEnumerationHandler, top_windows)
    for i in top_windows:
        if "notepad" in i[1].lower():
            print i
            win32gui.ShowWindow(i[0],5)
            win32gui.SetForegroundWindow(i[0])
            break

We only need PyWin32's win32gui module for this little script. We write a little function that takes a window handle and a Python list. Then we call win32gui's EnumWindows method, which takes a callback and an extra argument that is a Python object. According to the documentation, the EnumWindows method "Enumerates all top-level windows on the screen by passing the handle to each window, in turn, to an application-defined callback function". So we pass it our method and it enumerates the windows, passing a handle of each window plus our Python list to our function. It works kind of like a messed up decorator.

Once that's done, your top_windows list will be full of lots of items, most of which you didn't even know were running. You can print that our and inspect your results if you like. It's really quite intereting. But for our purposes, we will skip that and just loop over the list, looking for the word "Notepad". Once we find it, we use win32gui's ShowWindow and SetForegroundWindow methods to bring the application to the foreground.

Note that really need to look for a unique string so that you bring up the right window. What would happen if you had multiple Notepad instance running with different files open? With the current code, you would bring the first Notepad instance that it found forward, which might not be what you want.

You may be wondering why anyone would even want to go to the trouble of doing this in the first place. In my case, I once had a project where I had to bring a certain window to the foreground and enter automate it using SendKeys. It was an ugly piece of brittle code that I wouldn't wish on anyone. Fortunately, there are better tools for that sort of thing nowadays such as pywinauto, but you still might find this code helpful in something esoteric that is thrown your way. Have fun!

Note: This code was tested using Python 2.7.8 and PyWin32 219 on Windows 7.

20 Oct 2014 10:15pm GMT

Carl Trachte: subprocess.Popen() or Abusing a Home-grown Windows Executable

Each month I redo 3D block model interpolations for a series of open pits at a distant mine. Those of you who follow my twitter feed often see me tweet, "The 3D geologic block model interpolation chuggeth . . ." What's going on is that I've got all the processing power maxed out dealing with millions of model blocks and thousands of data points. The machine heats up and with the fan sounds like a DC-9 warming up before flight.

All that said, running everything roughly in parallel is more efficient time-wise than running it sequentially. An hour of chugging is better than four. The way I've been doing this is using the Python (2.7) subprocess module's Popen method, running my five interpolated values in parallel. Our Python programmer Lori originally wrote this to run in sequence for a different set of problems. I bastardized it for my own.

The subprocess part of the code is relatively straightforward. Function startprocess() in my code covers that.

What makes this problem a little more challenging:

1) it's a vendor supplied executable we're dealing with . . . without an API or source . . . that's interactive (you can't feed it the config file path; it asks for it). This results in a number of time.sleep() and <process>.stdin.write() calls that can be brittle.

2) getting the processes started, as I just mentioned, is easy. Finding out when to stop, or kill them, requires knowledge of the app and how it generates output. I've gone for an ugly, but effective check of report file contents.

3) while waiting for the processes to finish their work, I need to know things are working and what's going on. I've accomplished this by reporting the data files' sizes in MB.

4) the executable isn't designed for a centralized code base (typically all scripts are kept in a folder for the specific project or pit), so it only allows about 100 character columns in the file paths sent to it. I've omitted this from my sanitized version of the code, but it made things even messier than they are below. Also, I don't know if all Windows programs do this, but the paths need to be inside quotes - the path kept breaking on the colon (:) when not quoted.

Basically, this is a fairly ugly problem and a script that requires babysitting while it runs. That's OK; it beats the alternative (running it sequentially while watching each run). I've tried to adhere to DRY (don't repeat yourself) as much as possible, but I suspect this could be improved upon.

The reason why I blog it is that I suspect there are other people out there who have to do the same sort of thing with their data. It doesn't have to be a mining problem. It can be anything that requires intensive computation across voluminous data with an executable not designed with a Python API.

Notes:

1) I've omitted the file multirunparameters.py that's in an import statement. It has a bunch of paths and names that are relevant to my project, but not to the reader's programming needs.

2) python 2.7 is listed at the top of the file as "mpython." This is the Python that our mine planning vendor ships that ties into their quite capable Python API. The executable I call with subprocess.Popen() is a Windows executable provided by a consultant independent of the mine planning vendor. It just makes sense to package this interpolation inside the mine planning vendor's multirun (~ batch file) framework as part of an overall working of the 3D geologic block model. The script exits as soon as this part of the batch is complete. I've inserted a 10 second pause at the end just to allow a quick look before it disappears.

#!C:/MineSight/x64/mpython

"""
Interpolate grades with <consultant> program
from text files.
"""


import argparse
import subprocess as subx
import os
import collections as colx

import time
from datetime import datetime as dt


# Lookup file of constants, pit names, assay names, paths, etc.
import multirunparameters as paramsx


parser = argparse.ArgumentParser()
# 4 letter argument like 'kwat'
# Feed in at command line.
parser.add_argument('pit', help='four letter, lower case pit abbreviation (kwat)', type=str)
args = parser.parse_args()
PIT = args.pit


pitdir = paramsx.PATHS[PIT]
pathx = paramsx.BASEPATH.format(pitdir)
controlfilepathx = paramsx.CONTROLFILEPATH.format(pitdir)


timestart = dt.now()
print(timestart)


PROGRAM = 'C:/MSPROJECTS/EOMReconciliation/2014/Multirun/AllPits/consultantprogram.exe'

ENDTEXT = 'END <consultant> REPORT'

# These names are the only real difference between pits.
# Double quote is for subprocess.Popen object's stdin.write method
# - Windows path breaks on colon without quotes.
ASSAY1DRIVER = 'KDriverASSAY1{:s}CBT.csv"'.format(PIT)
ASSAY2DRIVER = 'KDriverASSAY2{:s}CBT.csv"'.format(PIT)
ASSAY3DRIVER = 'KDriverASSAY3_{:s}CBT.csv"'.format(PIT)
ASSAY4DRIVER = 'KDriverASSAY4_{:s}CBT.csv"'.format(PIT)
ASSAY5DRIVER = 'KDriverASSAY5_{:s}CBT.csv"'.format(PIT)


RETCHAR = '\n'

ASSAY1 = 'ASSAY1'
ASSAY2 = 'ASSAY2'
ASSAY3 = 'ASSAY3'
ASSAY4 = 'ASSAY4'
ASSAY5 = 'ASSAY5'


NAME = 'name'
DRFILE = 'driver file'
OUTPUT = 'output'
DATFILE = 'data file'
RPTFILE = 'report file'


# data, report files
ASSAY1K = 'ASSAY1K.csv'
ASSAY1RPT = 'ASSAY1.RPT'

ASSAY2K = 'ASSAY2K.csv'
ASSAY2RPT = 'ASSAY2.RPT'

ASSAY3K = 'ASSAY3K.csv'
ASSAY3RPT = 'ASSAY3.RPT'

ASSAY4K = 'ASSAY4K.csv'
ASSAY4RPT = 'ASSAY4.RPT'

ASSAY5K = 'ASSAY5K.csv'
ASSAY5RPT = 'ASSAY5.RPT'


OUTPUTFMT = '{:s}output.txt'

ASSAYS = {1:{NAME:ASSAY1,
DRFILE:controlfilepathx + ASSAY1DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY1),
DATFILE:pathx + ASSAY1K,
RPTFILE:pathx + ASSAY1RPT},
2:{NAME:ASSAY2,
DRFILE:controlfilepathx + ASSAY2DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY2),
DATFILE:pathx + ASSAY2K,
RPTFILE:pathx + ASSAY2RPT},
3:{NAME:ASSAY3,
DRFILE:controlfilepathx + ASSAY3DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY3),
DATFILE:pathx + ASSAY3K,
RPTFILE:pathx + ASSAY3RPT},
4:{NAME:ASSAY4,
DRFILE:controlfilepathx + ASSAY4DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY4),
DATFILE:pathx + ASSAY4K,
RPTFILE:pathx + ASSAY4RPT},
5:{NAME:ASSAY5,
DRFILE:controlfilepathx + ASSAY5DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY5),
DATFILE:pathx + ASSAY5K,
RPTFILE:pathx + ASSAY5RPT}}


DELFILE = 'delete file'
INTERP = 'interp'
SLEEP = 'sleep'
MSGDRIVER = 'message driver'
MSGRETCHAR = 'message return character'
FINISHED1 = 'finished one assay'
FINISHEDALL = 'finished all interpolations'
TIMEELAPSED = 'time elapsed'
FILEEXISTS = 'report file exists'
DATSIZE = 'data file size'
DONE = 'number interpolations finished'
DATFILEEXIST = 'data file not yet there'
SIZECHANGE = 'report file changed size'


# for converting to megabyte file size from os.stat()
BITSHIFT = 20

# sleeptime - 5 seconds
SLEEPTIME = 5

FINISHED = 'finished'
RPTFILECHSIZE = """

Report file for {:s}
changed size; killing process . . .

"""

MESGS = {DELFILE:'\n\nDeleting {} . . .\n\n',
INTERP:'\n\nInterpolating {:s} . . .\n\n',
SLEEP:'\nSleeping 2 seconds . . .\n\n',
MSGDRIVER:'\n\nWriting driver file name to stdin . . .\n\n',
MSGRETCHAR:'\n\nWriting retchar to stdin for {:s} . . .\n\n',
FINISHED1:'\n\nFinished {:s}\n\n',
FINISHEDALL:'\n\nFinished interpolation.\n\n',
TIMEELAPSED:'\n\n{:d} elapsed seconds\n\n',
FILEEXISTS:'\n\nReport file for {:s} exists . . .\n\n',
DATSIZE:'\n\nData file size for {:s} is now {:d}MB . . .\n\n',
DONE:'\n\n{:d} out of {:d} assays are finished . . .\n\n',
DATFILEEXIST:"\n\n{:s} doesn't exist yet . . .\n\n",
SIZECHANGE:RPTFILECHSIZE}


def cleanslate():
"""
Delete all output files prior to interpolation
so that their existence can be tracked.
"""
for key in ASSAYS:
files = (ASSAYS[key][DATFILE],
ASSAYS[key][RPTFILE],
ASSAYS[key][OUTPUT])
for filex in files:
print(MESGS[DELFILE].format(filex))
if os.path.exists(filex) and os.path.isfile(filex):
os.remove(filex)
return 0


def startprocess(assay):
"""
Start <consultant program> run for given interpolation.

Return subprocess.Popen object,
file object (output file).
"""
print(MESGS[INTERP].format(ASSAYS[assay][NAME]))
# XXX - I hate time.sleep - hack
# XXX - try to re-route standard output so that
# it's not all jumbled together.
print(MESGS[SLEEP])
time.sleep(2)
# output file for stdout
f = open(ASSAYS[assay][OUTPUT], 'w')
procx = subx.Popen('{0}'.format(PROGRAM), stdin=subx.PIPE, stdout=f)
print(MESGS[SLEEP])
time.sleep(2)
# XXX - problem, starting up Excel CBT 22JUN2014
# Ah - this is what happens when the <software usb licence>
# key is not attached :-(
print(MESGS[MSGDRIVER])
print('\ndriver file = {:s}\n'.format(ASSAYS[assay][DRFILE]))
procx.stdin.write(ASSAYS[assay][DRFILE])
print(MESGS[SLEEP])
time.sleep(2)
# XXX - this is so jacked up -
# no idea what is happening when
print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
procx.stdin.write(RETCHAR)
print(MESGS[SLEEP])
time.sleep(2)
print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
procx.stdin.write(RETCHAR)
print(MESGS[SLEEP])
time.sleep(2)
return procx, f


def crosslookup(assay):
"""
From assay string, get numeric
key for ASSAYS dictionary.

Returns integer.
"""
for key in ASSAYS:
if assay == ASSAYS[key][NAME]:
return key
return 0


def checkprocess(assay, assaydict):
"""
Check to see if assay
interpolation is finished.

assay is the item in question
(ASSAY1, ASSAY2, etc.).

assaydict is the operating dictionary
for the assay in question.

Returns True if finished.
"""
# Report file indicates process finished.
assaykey = crosslookup(assay)
rptfile = ASSAYS[assaykey][RPTFILE]
datfile = ASSAYS[assaykey][DATFILE]
if os.path.exists(datfile) and os.path.isfile(datfile):
# Report size of file in MB.
datfilesize = os.stat(datfile).st_size >> BITSHIFT
print(MESGS[DATSIZE].format(assay, datfilesize))
else:
# Doesn't exist yet.
print(MESGS[DATFILEEXIST].format(datfile))
if os.path.exists(rptfile) and os.path.isfile(rptfile):
# XXX - not the most efficient way,
# but this checking the file appears
# to work best.
f = open(rptfile, 'r')
txt = f.read()
f.close()
# XXX - hack - gah.
if txt.find(ENDTEXT) > -1:
# looking for change in reportfile size
# or big report file
print(MESGS[SIZECHANGE].format(assay))
print(MESGS[SLEEP])
time.sleep(2)
return True
return False


PROCX = 'process'
OUTPUTFILE = 'output file'


# Keeps track of files and progress of <consultant program>.
opdict = colx.OrderedDict()


# get rid of preexisting files
cleanslate()


# start all five roughly in parallel
# ASSAYS keys are numbers
for key in ASSAYS:
# opdict - ordered with assay names as keys
namex = ASSAYS[key][NAME]
opdict[namex] = {}
assaydict = opdict[namex]
assaydict[PROCX], assaydict[OUTPUTFILE] = startprocess(key)
# Initialize active status of process.
assaydict[FINISHED] = False


# For count.
numassays = len(ASSAYS)
# Loop until all finished.
while True:
# Cycle until done then break.
# Sleep SLEEPTIME seconds at a time and check between.
time.sleep(SLEEPTIME)
# Count.
i = 0
for key in opdict:
assaydict = opdict[key]
if not assaydict[FINISHED]:
status = checkprocess(key, assaydict)
if status:
# kill process when report file changes
opdict[key][PROCX].kill()
assaydict[FINISHED] = True
i += 1
else:
i += 1
print(MESGS[DONE].format(i, numassays))
# all done
if i == numassays:
break


print('\n\nFinished interpolation.\n\n')
timeend = dt.now()
elapsed = timeend - timestart


print(MESGS[TIMEELAPSED].format(elapsed.seconds))
print('\n\n{:d} elapsed minutes\n\n'.format(elapsed.seconds/60))


# Allow quick look at screen.
time.sleep(10)



20 Oct 2014 9:11pm GMT

Carl Trachte: subprocess.Popen() or Abusing a Home-grown Windows Executable

Each month I redo 3D block model interpolations for a series of open pits at a distant mine. Those of you who follow my twitter feed often see me tweet, "The 3D geologic block model interpolation chuggeth . . ." What's going on is that I've got all the processing power maxed out dealing with millions of model blocks and thousands of data points. The machine heats up and with the fan sounds like a DC-9 warming up before flight.

All that said, running everything roughly in parallel is more efficient time-wise than running it sequentially. An hour of chugging is better than four. The way I've been doing this is using the Python (2.7) subprocess module's Popen method, running my five interpolated values in parallel. Our Python programmer Lori originally wrote this to run in sequence for a different set of problems. I bastardized it for my own.

The subprocess part of the code is relatively straightforward. Function startprocess() in my code covers that.

What makes this problem a little more challenging:

1) it's a vendor supplied executable we're dealing with . . . without an API or source . . . that's interactive (you can't feed it the config file path; it asks for it). This results in a number of time.sleep() and <process>.stdin.write() calls that can be brittle.

2) getting the processes started, as I just mentioned, is easy. Finding out when to stop, or kill them, requires knowledge of the app and how it generates output. I've gone for an ugly, but effective check of report file contents.

3) while waiting for the processes to finish their work, I need to know things are working and what's going on. I've accomplished this by reporting the data files' sizes in MB.

4) the executable isn't designed for a centralized code base (typically all scripts are kept in a folder for the specific project or pit), so it only allows about 100 character columns in the file paths sent to it. I've omitted this from my sanitized version of the code, but it made things even messier than they are below. Also, I don't know if all Windows programs do this, but the paths need to be inside quotes - the path kept breaking on the colon (:) when not quoted.

Basically, this is a fairly ugly problem and a script that requires babysitting while it runs. That's OK; it beats the alternative (running it sequentially while watching each run). I've tried to adhere to DRY (don't repeat yourself) as much as possible, but I suspect this could be improved upon.

The reason why I blog it is that I suspect there are other people out there who have to do the same sort of thing with their data. It doesn't have to be a mining problem. It can be anything that requires intensive computation across voluminous data with an executable not designed with a Python API.

Notes:

1) I've omitted the file multirunparameters.py that's in an import statement. It has a bunch of paths and names that are relevant to my project, but not to the reader's programming needs.

2) python 2.7 is listed at the top of the file as "mpython." This is the Python that our mine planning vendor ships that ties into their quite capable Python API. The executable I call with subprocess.Popen() is a Windows executable provided by a consultant independent of the mine planning vendor. It just makes sense to package this interpolation inside the mine planning vendor's multirun (~ batch file) framework as part of an overall working of the 3D geologic block model. The script exits as soon as this part of the batch is complete. I've inserted a 10 second pause at the end just to allow a quick look before it disappears.

#!C:/MineSight/x64/mpython

"""
Interpolate grades with <consultant> program
from text files.
"""


import argparse
import subprocess as subx
import os
import collections as colx

import time
from datetime import datetime as dt


# Lookup file of constants, pit names, assay names, paths, etc.
import multirunparameters as paramsx


parser = argparse.ArgumentParser()
# 4 letter argument like 'kwat'
# Feed in at command line.
parser.add_argument('pit', help='four letter, lower case pit abbreviation (kwat)', type=str)
args = parser.parse_args()
PIT = args.pit


pitdir = paramsx.PATHS[PIT]
pathx = paramsx.BASEPATH.format(pitdir)
controlfilepathx = paramsx.CONTROLFILEPATH.format(pitdir)


timestart = dt.now()
print(timestart)


PROGRAM = 'C:/MSPROJECTS/EOMReconciliation/2014/Multirun/AllPits/consultantprogram.exe'

ENDTEXT = 'END <consultant> REPORT'

# These names are the only real difference between pits.
# Double quote is for subprocess.Popen object's stdin.write method
# - Windows path breaks on colon without quotes.
ASSAY1DRIVER = 'KDriverASSAY1{:s}CBT.csv"'.format(PIT)
ASSAY2DRIVER = 'KDriverASSAY2{:s}CBT.csv"'.format(PIT)
ASSAY3DRIVER = 'KDriverASSAY3_{:s}CBT.csv"'.format(PIT)
ASSAY4DRIVER = 'KDriverASSAY4_{:s}CBT.csv"'.format(PIT)
ASSAY5DRIVER = 'KDriverASSAY5_{:s}CBT.csv"'.format(PIT)


RETCHAR = '\n'

ASSAY1 = 'ASSAY1'
ASSAY2 = 'ASSAY2'
ASSAY3 = 'ASSAY3'
ASSAY4 = 'ASSAY4'
ASSAY5 = 'ASSAY5'


NAME = 'name'
DRFILE = 'driver file'
OUTPUT = 'output'
DATFILE = 'data file'
RPTFILE = 'report file'


# data, report files
ASSAY1K = 'ASSAY1K.csv'
ASSAY1RPT = 'ASSAY1.RPT'

ASSAY2K = 'ASSAY2K.csv'
ASSAY2RPT = 'ASSAY2.RPT'

ASSAY3K = 'ASSAY3K.csv'
ASSAY3RPT = 'ASSAY3.RPT'

ASSAY4K = 'ASSAY4K.csv'
ASSAY4RPT = 'ASSAY4.RPT'

ASSAY5K = 'ASSAY5K.csv'
ASSAY5RPT = 'ASSAY5.RPT'


OUTPUTFMT = '{:s}output.txt'

ASSAYS = {1:{NAME:ASSAY1,
DRFILE:controlfilepathx + ASSAY1DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY1),
DATFILE:pathx + ASSAY1K,
RPTFILE:pathx + ASSAY1RPT},
2:{NAME:ASSAY2,
DRFILE:controlfilepathx + ASSAY2DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY2),
DATFILE:pathx + ASSAY2K,
RPTFILE:pathx + ASSAY2RPT},
3:{NAME:ASSAY3,
DRFILE:controlfilepathx + ASSAY3DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY3),
DATFILE:pathx + ASSAY3K,
RPTFILE:pathx + ASSAY3RPT},
4:{NAME:ASSAY4,
DRFILE:controlfilepathx + ASSAY4DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY4),
DATFILE:pathx + ASSAY4K,
RPTFILE:pathx + ASSAY4RPT},
5:{NAME:ASSAY5,
DRFILE:controlfilepathx + ASSAY5DRIVER,
OUTPUT:pathx + OUTPUTFMT.format(ASSAY5),
DATFILE:pathx + ASSAY5K,
RPTFILE:pathx + ASSAY5RPT}}


DELFILE = 'delete file'
INTERP = 'interp'
SLEEP = 'sleep'
MSGDRIVER = 'message driver'
MSGRETCHAR = 'message return character'
FINISHED1 = 'finished one assay'
FINISHEDALL = 'finished all interpolations'
TIMEELAPSED = 'time elapsed'
FILEEXISTS = 'report file exists'
DATSIZE = 'data file size'
DONE = 'number interpolations finished'
DATFILEEXIST = 'data file not yet there'
SIZECHANGE = 'report file changed size'


# for converting to megabyte file size from os.stat()
BITSHIFT = 20

# sleeptime - 5 seconds
SLEEPTIME = 5

FINISHED = 'finished'
RPTFILECHSIZE = """

Report file for {:s}
changed size; killing process . . .

"""

MESGS = {DELFILE:'\n\nDeleting {} . . .\n\n',
INTERP:'\n\nInterpolating {:s} . . .\n\n',
SLEEP:'\nSleeping 2 seconds . . .\n\n',
MSGDRIVER:'\n\nWriting driver file name to stdin . . .\n\n',
MSGRETCHAR:'\n\nWriting retchar to stdin for {:s} . . .\n\n',
FINISHED1:'\n\nFinished {:s}\n\n',
FINISHEDALL:'\n\nFinished interpolation.\n\n',
TIMEELAPSED:'\n\n{:d} elapsed seconds\n\n',
FILEEXISTS:'\n\nReport file for {:s} exists . . .\n\n',
DATSIZE:'\n\nData file size for {:s} is now {:d}MB . . .\n\n',
DONE:'\n\n{:d} out of {:d} assays are finished . . .\n\n',
DATFILEEXIST:"\n\n{:s} doesn't exist yet . . .\n\n",
SIZECHANGE:RPTFILECHSIZE}


def cleanslate():
"""
Delete all output files prior to interpolation
so that their existence can be tracked.
"""
for key in ASSAYS:
files = (ASSAYS[key][DATFILE],
ASSAYS[key][RPTFILE],
ASSAYS[key][OUTPUT])
for filex in files:
print(MESGS[DELFILE].format(filex))
if os.path.exists(filex) and os.path.isfile(filex):
os.remove(filex)
return 0


def startprocess(assay):
"""
Start <consultant program> run for given interpolation.

Return subprocess.Popen object,
file object (output file).
"""
print(MESGS[INTERP].format(ASSAYS[assay][NAME]))
# XXX - I hate time.sleep - hack
# XXX - try to re-route standard output so that
# it's not all jumbled together.
print(MESGS[SLEEP])
time.sleep(2)
# output file for stdout
f = open(ASSAYS[assay][OUTPUT], 'w')
procx = subx.Popen('{0}'.format(PROGRAM), stdin=subx.PIPE, stdout=f)
print(MESGS[SLEEP])
time.sleep(2)
# XXX - problem, starting up Excel CBT 22JUN2014
# Ah - this is what happens when the <software usb licence>
# key is not attached :-(
print(MESGS[MSGDRIVER])
print('\ndriver file = {:s}\n'.format(ASSAYS[assay][DRFILE]))
procx.stdin.write(ASSAYS[assay][DRFILE])
print(MESGS[SLEEP])
time.sleep(2)
# XXX - this is so jacked up -
# no idea what is happening when
print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
procx.stdin.write(RETCHAR)
print(MESGS[SLEEP])
time.sleep(2)
print(MESGS[MSGRETCHAR].format(ASSAYS[assay][NAME]))
procx.stdin.write(RETCHAR)
print(MESGS[SLEEP])
time.sleep(2)
return procx, f


def crosslookup(assay):
"""
From assay string, get numeric
key for ASSAYS dictionary.

Returns integer.
"""
for key in ASSAYS:
if assay == ASSAYS[key][NAME]:
return key
return 0


def checkprocess(assay, assaydict):
"""
Check to see if assay
interpolation is finished.

assay is the item in question
(ASSAY1, ASSAY2, etc.).

assaydict is the operating dictionary
for the assay in question.

Returns True if finished.
"""
# Report file indicates process finished.
assaykey = crosslookup(assay)
rptfile = ASSAYS[assaykey][RPTFILE]
datfile = ASSAYS[assaykey][DATFILE]
if os.path.exists(datfile) and os.path.isfile(datfile):
# Report size of file in MB.
datfilesize = os.stat(datfile).st_size >> BITSHIFT
print(MESGS[DATSIZE].format(assay, datfilesize))
else:
# Doesn't exist yet.
print(MESGS[DATFILEEXIST].format(datfile))
if os.path.exists(rptfile) and os.path.isfile(rptfile):
# XXX - not the most efficient way,
# but this checking the file appears
# to work best.
f = open(rptfile, 'r')
txt = f.read()
f.close()
# XXX - hack - gah.
if txt.find(ENDTEXT) > -1:
# looking for change in reportfile size
# or big report file
print(MESGS[SIZECHANGE].format(assay))
print(MESGS[SLEEP])
time.sleep(2)
return True
return False


PROCX = 'process'
OUTPUTFILE = 'output file'


# Keeps track of files and progress of <consultant program>.
opdict = colx.OrderedDict()


# get rid of preexisting files
cleanslate()


# start all five roughly in parallel
# ASSAYS keys are numbers
for key in ASSAYS:
# opdict - ordered with assay names as keys
namex = ASSAYS[key][NAME]
opdict[namex] = {}
assaydict = opdict[namex]
assaydict[PROCX], assaydict[OUTPUTFILE] = startprocess(key)
# Initialize active status of process.
assaydict[FINISHED] = False


# For count.
numassays = len(ASSAYS)
# Loop until all finished.
while True:
# Cycle until done then break.
# Sleep SLEEPTIME seconds at a time and check between.
time.sleep(SLEEPTIME)
# Count.
i = 0
for key in opdict:
assaydict = opdict[key]
if not assaydict[FINISHED]:
status = checkprocess(key, assaydict)
if status:
# kill process when report file changes
opdict[key][PROCX].kill()
assaydict[FINISHED] = True
i += 1
else:
i += 1
print(MESGS[DONE].format(i, numassays))
# all done
if i == numassays:
break


print('\n\nFinished interpolation.\n\n')
timeend = dt.now()
elapsed = timeend - timestart


print(MESGS[TIMEELAPSED].format(elapsed.seconds))
print('\n\n{:d} elapsed minutes\n\n'.format(elapsed.seconds/60))


# Allow quick look at screen.
time.sleep(10)



20 Oct 2014 9:11pm GMT

Tryton News: New Tryton release 3.4

We are proud to announce the 3.4 release of Tryton.

In addition to the usual improvements of existing features for users and developers, this release has seen a lot of work done on the accounting part.

Of course, migration from previous series is fully supported with the obvious exception of the ldap_connection module which was removed.

Major changes in graphical user interface

  • The search of relation record has been re-worked to take advantage of the auto-completion. The search box of the pop-up window is filled with the text entered in the widget.

  • The search/open button of the Many2One widget is now inside the entry box and the create button is removed in favor of auto-completion actions or pop-up button. This change allow to harmonize the size of all widgets inside a form.

    many2one button inside
  • A new image widget is available on list/tree view.

    widget image tree
  • The client can now perform a pre-validation before executing a button action. The validation is based on a domain and so the offending fields can be highlighted and focused instead of having an error message pop-up.

  • The selection label are now available in addition of the internal value for the export data (CSV) functionality.

  • The export data window is now predefined with the fields of the current view. This gives a fast way to export what you see.

  • The predefined export can now be replaced directly with a new selection of fields. This eases the process of creating such predefined exportation.

  • It is now possible to re-order the list of the exported fields using drag and drop.

  • The range operator of the search box is now including on both endpoints. It appears to be less astonishing behavior for users even if the previous behavior including-excluding had some practical advantages.

  • The client loads now plug-ins defined in the user local directory (~/.config/tryton/x.y/plugins).

Major changes on the server side

  • A new Mixin MatchMixin is introduced. It allows to implement a common pattern in Tryton to find records that match certain values.
  • Another UnionMixin is also introduced. It allows to define a ModelSQL which is the UNION of some ModelSQL's.
  • Actually, Tryton doesn't update a record defined in a XML file if this one has been modified outside the XML. Now, it is possible to find those records and force the update to get the record synchronised with the XML.
  • A Python descriptor has been added to the Selection field. It allows to define an attribute on a Model which will contains the selection label of the record. It is planned to update all the reports to use such descriptor instead of hard-coded values.
  • A new configuration file format is introduced for the server. It is easily extendable to be used by modules. For example, the ldap_authentication module starts using it in replacement of the removed ldap_connection.
  • It is now possible to give a logging configuration files to setup the server logging. This file uses the Python logging configuration format.
  • The context defined on relation fields are now used to instantiate the target.
  • The SQL clause for a domain on a field can be now customized using a domain_<field> method. This method allows in some cases a more efficient SQL query. The method is designed to support joins.
  • The access rights has been reworked to be active only on RPC calls. With this design, Tryton follows the principle of checking input on the border of the application. So it is no more required to switch to the root user when calling methods requiring some specific access rights as far as it is not from an RPC call.

Modules

Account

  • A new wizard to help reconcile all accounts has been added. It loops over each account and party and makes a proposal of lines to reconcile if it could find one. This really speeds up the reconciliation task.

    reoncilie wizard
  • There is also another new wizard to ease the creation of cancellation moves. The wizard also reconciles automatically the line with the cancelled sibling.

  • A new option Party Required on account has been added. This option makes the party required for move lines of this account and forbids it for others.

Account Invoice

  • It is now possible to configure which tax rounding to use. There are two ways implemented: per document and per line. The default stays per document.

Account Payment

  • It is now possible to change a succeeded payment to failed.

Account Payment SEPA

  • The scheme Business to Business is supported for direct debit.
  • The mandate receives now a default unique identification using a configured sequence.
  • The module supports now the bank to customer debit/credit notification message (CAMT.054).
  • A report to print a standard form for mandate has been added.

Account Statement

  • It is now possible to order the statement lines and to give them a number. With those features, it is easier to reproduce the same layout of a bank statement.
  • A report for statement has been added. For example, it can be used when using the statement for check deposit.
  • A validation method can be defined on the statement journal. The available methods are: Balance, Amount and Number of Lines. This helps to uses the statement for different purposes like bank statement or check deposit.

Account Stock Continental/Anglo-Saxon

  • The method is now defined on the fiscal year instead of being globally activated on module installation.

Country

  • It is now possible to store zip code per country. A script is provided to load zip codes from GeoNames.

LDAP Authentication

  • The module ldap_connection has been replaced by an entry in the configuration file of trytond.

Party

  • The new zip code from the module country is used to auto-complete zip and city field on address.

Purchase

  • The Confirmed state has been split into Confirmed and Processing, just like the Sale workflow.

Sale Supply Drop Shipment

  • The management of exception on drop shipment is propagated from the sale to the purchase.

New modules

  • The Account Payment Clearing module allows to generate clearing account move when a payment has succeeded between the receivable/payable account to a clearing account. The clearing account will be reconciled later by the statement.

Proteus

Proteus is a library to access Tryton like a client.

  • It is now possible to run reports. It is useful for testing them.
  • A new duplicate method is added which is similar to the copy menu entry of the client.

20 Oct 2014 6:00pm GMT

Tryton News: New Tryton release 3.4

We are proud to announce the 3.4 release of Tryton.

In addition to the usual improvements of existing features for users and developers, this release has seen a lot of work done on the accounting part.

Of course, migration from previous series is fully supported with the obvious exception of the ldap_connection module which was removed.

Major changes in graphical user interface

  • The search of relation record has been re-worked to take advantage of the auto-completion. The search box of the pop-up window is filled with the text entered in the widget.

  • The search/open button of the Many2One widget is now inside the entry box and the create button is removed in favor of auto-completion actions or pop-up button. This change allow to harmonize the size of all widgets inside a form.

    many2one button inside
  • A new image widget is available on list/tree view.

    widget image tree
  • The client can now perform a pre-validation before executing a button action. The validation is based on a domain and so the offending fields can be highlighted and focused instead of having an error message pop-up.

  • The selection label are now available in addition of the internal value for the export data (CSV) functionality.

  • The export data window is now predefined with the fields of the current view. This gives a fast way to export what you see.

  • The predefined export can now be replaced directly with a new selection of fields. This eases the process of creating such predefined exportation.

  • It is now possible to re-order the list of the exported fields using drag and drop.

  • The range operator of the search box is now including on both endpoints. It appears to be less astonishing behavior for users even if the previous behavior including-excluding had some practical advantages.

  • The client loads now plug-ins defined in the user local directory (~/.config/tryton/x.y/plugins).

Major changes on the server side

  • A new Mixin MatchMixin is introduced. It allows to implement a common pattern in Tryton to find records that match certain values.
  • Another UnionMixin is also introduced. It allows to define a ModelSQL which is the UNION of some ModelSQL's.
  • Actually, Tryton doesn't update a record defined in a XML file if this one has been modified outside the XML. Now, it is possible to find those records and force the update to get the record synchronised with the XML.
  • A Python descriptor has been added to the Selection field. It allows to define an attribute on a Model which will contains the selection label of the record. It is planned to update all the reports to use such descriptor instead of hard-coded values.
  • A new configuration file format is introduced for the server. It is easily extendable to be used by modules. For example, the ldap_authentication module starts using it in replacement of the removed ldap_connection.
  • It is now possible to give a logging configuration files to setup the server logging. This file uses the Python logging configuration format.
  • The context defined on relation fields are now used to instantiate the target.
  • The SQL clause for a domain on a field can be now customized using a domain_<field> method. This method allows in some cases a more efficient SQL query. The method is designed to support joins.
  • The access rights has been reworked to be active only on RPC calls. With this design, Tryton follows the principle of checking input on the border of the application. So it is no more required to switch to the root user when calling methods requiring some specific access rights as far as it is not from an RPC call.

Modules

Account

  • A new wizard to help reconcile all accounts has been added. It loops over each account and party and makes a proposal of lines to reconcile if it could find one. This really speeds up the reconciliation task.

    reoncilie wizard
  • There is also another new wizard to ease the creation of cancellation moves. The wizard also reconciles automatically the line with the cancelled sibling.

  • A new option Party Required on account has been added. This option makes the party required for move lines of this account and forbids it for others.

Account Invoice

  • It is now possible to configure which tax rounding to use. There are two ways implemented: per document and per line. The default stays per document.

Account Payment

  • It is now possible to change a succeeded payment to failed.

Account Payment SEPA

  • The scheme Business to Business is supported for direct debit.
  • The mandate receives now a default unique identification using a configured sequence.
  • The module supports now the bank to customer debit/credit notification message (CAMT.054).
  • A report to print a standard form for mandate has been added.

Account Statement

  • It is now possible to order the statement lines and to give them a number. With those features, it is easier to reproduce the same layout of a bank statement.
  • A report for statement has been added. For example, it can be used when using the statement for check deposit.
  • A validation method can be defined on the statement journal. The available methods are: Balance, Amount and Number of Lines. This helps to uses the statement for different purposes like bank statement or check deposit.

Account Stock Continental/Anglo-Saxon

  • The method is now defined on the fiscal year instead of being globally activated on module installation.

Country

  • It is now possible to store zip code per country. A script is provided to load zip codes from GeoNames.

LDAP Authentication

  • The module ldap_connection has been replaced by an entry in the configuration file of trytond.

Party

  • The new zip code from the module country is used to auto-complete zip and city field on address.

Purchase

  • The Confirmed state has been split into Confirmed and Processing, just like the Sale workflow.

Sale Supply Drop Shipment

  • The management of exception on drop shipment is propagated from the sale to the purchase.

New modules

  • The Account Payment Clearing module allows to generate clearing account move when a payment has succeeded between the receivable/payable account to a clearing account. The clearing account will be reconciled later by the statement.

Proteus

Proteus is a library to access Tryton like a client.

  • It is now possible to run reports. It is useful for testing them.
  • A new duplicate method is added which is similar to the copy menu entry of the client.

20 Oct 2014 6:00pm GMT

10 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: King Willams Town Bahnhof

Gestern musste ich morgens zur Station nach KWT um unsere Rerservierten Bustickets für die Weihnachtsferien in Capetown abzuholen. Der Bahnhof selber ist seit Dezember aus kostengründen ohne Zugverbindung - aber Translux und co - die langdistanzbusse haben dort ihre Büros.


Größere Kartenansicht




© benste CC NC SA

10 Nov 2011 10:57am GMT

09 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein

Niemand ist besorgt um so was - mit dem Auto fährt man einfach durch, und in der City - nahe Gnobie- "ne das ist erst gefährlich wenn die Feuerwehr da ist" - 30min später auf dem Rückweg war die Feuerwehr da.




© benste CC NC SA

09 Nov 2011 8:25pm GMT

08 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Brai Party

Brai = Grillabend o.ä.

Die möchte gern Techniker beim Flicken ihrer SpeakOn / Klinke Stecker Verzweigungen...

Die Damen "Mamas" der Siedlung bei der offiziellen Eröffnungsrede

Auch wenn weniger Leute da waren als erwartet, Laute Musik und viele Leute ...

Und natürlich ein Feuer mit echtem Holz zum Grillen.

© benste CC NC SA

08 Nov 2011 2:30pm GMT

07 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Lumanyano Primary

One of our missions was bringing Katja's Linux Server back to her room. While doing that we saw her new decoration.

Björn, Simphiwe carried the PC to Katja's school


© benste CC NC SA

07 Nov 2011 2:00pm GMT

06 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nelisa Haircut

Today I went with Björn to Needs Camp to Visit Katja's guest family for a special Party. First of all we visited some friends of Nelisa - yeah the one I'm working with in Quigney - Katja's guest fathers sister - who did her a haircut.

African Women usually get their hair done by arranging extensions and not like Europeans just cutting some hair.

In between she looked like this...

And then she was done - looks amazing considering the amount of hair she had last week - doesn't it ?

© benste CC NC SA

06 Nov 2011 7:45pm GMT

05 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Mein Samstag

Irgendwie viel mir heute auf das ich meine Blogposts mal ein bischen umstrukturieren muss - wenn ich immer nur von neuen Plätzen berichte, dann müsste ich ja eine Rundreise machen. Hier also mal ein paar Sachen aus meinem heutigen Alltag.

Erst einmal vorweg, Samstag zählt zumindest für uns Voluntäre zu den freien Tagen.

Dieses Wochenende sind nur Rommel und ich auf der Farm - Katja und Björn sind ja mittlerweile in ihren Einsatzstellen, und meine Mitbewohner Kyle und Jonathan sind zu Hause in Grahamstown - sowie auch Sipho der in Dimbaza wohnt.
Robin, die Frau von Rommel ist in Woodie Cape - schon seit Donnerstag um da ein paar Sachen zur erledigen.
Naja wie dem auch sei heute morgen haben wir uns erstmal ein gemeinsames Weetbix/Müsli Frühstück gegönnt und haben uns dann auf den Weg nach East London gemacht. 2 Sachen waren auf der Checkliste Vodacom, Ethienne (Imobilienmakler) außerdem auf dem Rückweg die fehlenden Dinge nach NeedsCamp bringen.

Nachdem wir gerade auf der Dirtroad losgefahren sind mussten wir feststellen das wir die Sachen für Needscamp und Ethienne nicht eingepackt hatten aber die Pumpe für die Wasserversorgung im Auto hatten.

Also sind wir in EastLondon ersteinmal nach Farmerama - nein nicht das onlinespiel farmville - sondern einen Laden mit ganz vielen Sachen für eine Farm - in Berea einem nördlichen Stadteil gefahren.

In Farmerama haben wir uns dann beraten lassen für einen Schnellverschluss der uns das leben mit der Pumpe leichter machen soll und außerdem eine leichtere Pumpe zur Reperatur gebracht, damit es nicht immer so ein großer Aufwand ist, wenn mal wieder das Wasser ausgegangen ist.

Fego Caffé ist in der Hemmingways Mall, dort mussten wir und PIN und PUK einer unserer Datensimcards geben lassen, da bei der PIN Abfrage leider ein zahlendreher unterlaufen ist. Naja auf jeden Fall speichern die Shops in Südafrika so sensible Daten wie eine PUK - die im Prinzip zugang zu einem gesperrten Phone verschafft.

Im Cafe hat Rommel dann ein paar online Transaktionen mit dem 3G Modem durchgeführt, welches ja jetzt wieder funktionierte - und übrigens mittlerweile in Ubuntu meinem Linuxsystem perfekt klappt.

Nebenbei bin ich nach 8ta gegangen um dort etwas über deren neue Deals zu erfahren, da wir in einigen von Hilltops Centern Internet anbieten wollen. Das Bild zeigt die Abdeckung UMTS in NeedsCamp Katjas Ort. 8ta ist ein neuer Telefonanbieter von Telkom, nachdem Vodafone sich Telkoms anteile an Vodacom gekauft hat müssen die komplett neu aufbauen.
Wir haben uns dazu entschieden mal eine kostenlose Prepaidkarte zu testen zu organisieren, denn wer weis wie genau die Karte oben ist ... Bevor man einen noch so billigen Deal für 24 Monate signed sollte man wissen obs geht.

Danach gings nach Checkers in Vincent, gesucht wurden zwei Hotplates für WoodyCape - R 129.00 eine - also ca. 12€ für eine zweigeteilte Kochplatte.
Wie man sieht im Hintergrund gibts schon Weihnachtsdeko - Anfang November und das in Südafrika bei sonnig warmen min- 25°C

Mittagessen haben wir uns bei einem Pakistanischen Curry Imbiss gegönnt - sehr empfehlenswert !
Naja und nachdem wir dann vor ner Stunde oder so zurück gekommen sind habe ich noch den Kühlschrank geputzt den ich heute morgen zum defrosten einfach nach draußen gestellt hatte. Jetzt ist der auch mal wieder sauber und ohne 3m dicke Eisschicht...

Morgen ... ja darüber werde ich gesondert berichten ... aber vermutlich erst am Montag, denn dann bin ich nochmal wieder in Quigney(East London) und habe kostenloses Internet.

© benste CC NC SA

05 Nov 2011 4:33pm GMT

31 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Sterkspruit Computer Center

Sterkspruit is one of Hilltops Computer Centres in the far north of Eastern Cape. On the trip to J'burg we've used the opportunity to take a look at the centre.

Pupils in the big classroom


The Trainer


School in Countryside


Adult Class in the Afternoon


"Town"


© benste CC NC SA

31 Oct 2011 4:58pm GMT

Benedict Stein: Technical Issues

What are you doing in an internet cafe if your ADSL and Faxline has been discontinued before months end. Well my idea was sitting outside and eating some ice cream.
At least it's sunny and not as rainy as on the weekend.


© benste CC NC SA

31 Oct 2011 3:11pm GMT

30 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nellis Restaurant

For those who are traveling through Zastron - there is a very nice Restaurant which is serving delicious food at reasanable prices.
In addition they're selling home made juices jams and honey.




interior


home made specialities - the shop in the shop


the Bar


© benste CC NC SA

30 Oct 2011 4:47pm GMT

29 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: The way back from J'burg

Having the 10 - 12h trip from J'burg back to ELS I was able to take a lot of pcitures including these different roadsides

Plain Street


Orange River in its beginngings (near Lesotho)


Zastron Anglican Church


The Bridge in Between "Free State" and Eastern Cape next to Zastron


my new Background ;)


If you listen to GoogleMaps you'll end up traveling 50km of gravel road - as it was just renewed we didn't have that many problems and saved 1h compared to going the official way with all it's constructions sites




Freeway


getting dark


© benste CC NC SA

29 Oct 2011 4:23pm GMT

28 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Wie funktioniert eigentlich eine Baustelle ?

Klar einiges mag anders sein, vieles aber gleich - aber ein in Deutschland täglich übliches Bild einer Straßenbaustelle - wie läuft das eigentlich in Südafrika ?

Ersteinmal vorweg - NEIN keine Ureinwohner die mit den Händen graben - auch wenn hier mehr Manpower genutzt wird - sind sie fleißig mit Technologie am arbeiten.

Eine ganz normale "Bundesstraße"


und wie sie erweitert wird


gaaaanz viele LKWs


denn hier wird eine Seite über einen langen Abschnitt komplett gesperrt, so das eine Ampelschaltung mit hier 45 Minuten Wartezeit entsteht


Aber wenigstens scheinen die ihren Spaß zu haben ;) - Wie auch wir denn gücklicher Weise mussten wir nie länger als 10 min. warten.

© benste CC NC SA

28 Oct 2011 4:20pm GMT