25 Jan 2015

feedPlanet Python

Glyph Lefkowitz: Security as Stencil

Image Credit: Horia Varlan

On the Internet, it's important to secure all of your communications.

There are a number of applications which purport to give you "secure chat", "secure email", or "secure phone calls".

The problem with these applications is that they advertise their presence. Since "insecure chat", "insecure email" and "insecure phone calls" all have a particular, detectable signature, an interested observer may easily detect your supposedly "secure" communication. Not only that, but the places that you go to obtain them are suspicious in their own right. In order to visit Whisper Systems, you have to be looking for "secure" communications.

This allows the adversary to use "security" technologies such as encryption as a sort of stencil, to outline and highlight the communication that they really want to be capturing. In the case of the NSA, this dumps anyone who would like to have a serious private conversation with a friend into the same bucket, from the perspective of the authorities, as a conspiracy of psychopaths trying to commit mass murder.

The Snowden documents already demonstrate that the NSA does exactly this; if you send a normal email, they will probably lose interest and ignore it after a little while, whereas if you send a "secure" email, they will store it forever and keep trying to crack it to see what you're hiding.

If you're running a supposedly innocuous online service or writing a supposedly harmless application, the hassle associated with setting up TLS certificates and encryption keys may seem like a pointless distraction. It isn't.

For one thing, if you have anywhere that user-created content enters your service, you don't know what they are going to be using it to communicate. Maybe you're just writing an online game but users will use your game for something as personal as courtship. Can we agree that the state security services shouldn't be involved in that?. Even if you were specifically writing an app for dating, you might not anticipate that the police will use it to show up and arrest your users so that they will be savagely beaten in jail.

The technology problems that "secure" services are working on are all important. But we can't simply develop a good "secure" technology, consider it a niche product, and leave it at that. Those of us who are software development professionals need to build security into every product, because users expect it. Users expect it because we are, in a million implicit ways, telling them that they have it. If we put a "share with your friend!" button into a user interface, that's a claim: we're claiming that the data the user indicates is being shared only with their friend. Would we want to put in a button that says "share with your friend, and with us, and with the state security apparatus, and with any criminal who can break in and steal our database!"? Obviously not. So let's stop making the "share with your friend!" button actually do that.

Those of us who understand the importance of security and are in the business of creating secure software must, therefore, take on the Sisyphean task of not only creating good security, but of competing with the insecure software on its own turf, so that people actually use it. "Slightly worse to use than your regular email program, but secure" is not good enough. (Not to mention the fact that existing security solutions are more than "slightly" worse to use). Secure stuff has to be as good as or better than its insecure competitors.

I know that this is a monumental undertaking. I have personally tried and failed to do something like this more than once. As the Rabbi Tarfon put it, though:

It is not incumbent upon you to complete the work, but neither are you at liberty to desist from it.

25 Jan 2015 12:16am GMT

Glyph Lefkowitz: Security as Stencil

Image Credit: Horia Varlan

On the Internet, it's important to secure all of your communications.

There are a number of applications which purport to give you "secure chat", "secure email", or "secure phone calls".

The problem with these applications is that they advertise their presence. Since "insecure chat", "insecure email" and "insecure phone calls" all have a particular, detectable signature, an interested observer may easily detect your supposedly "secure" communication. Not only that, but the places that you go to obtain them are suspicious in their own right. In order to visit Whisper Systems, you have to be looking for "secure" communications.

This allows the adversary to use "security" technologies such as encryption as a sort of stencil, to outline and highlight the communication that they really want to be capturing. In the case of the NSA, this dumps anyone who would like to have a serious private conversation with a friend into the same bucket, from the perspective of the authorities, as a conspiracy of psychopaths trying to commit mass murder.

The Snowden documents already demonstrate that the NSA does exactly this; if you send a normal email, they will probably lose interest and ignore it after a little while, whereas if you send a "secure" email, they will store it forever and keep trying to crack it to see what you're hiding.

If you're running a supposedly innocuous online service or writing a supposedly harmless application, the hassle associated with setting up TLS certificates and encryption keys may seem like a pointless distraction. It isn't.

For one thing, if you have anywhere that user-created content enters your service, you don't know what they are going to be using it to communicate. Maybe you're just writing an online game but users will use your game for something as personal as courtship. Can we agree that the state security services shouldn't be involved in that?. Even if you were specifically writing an app for dating, you might not anticipate that the police will use it to show up and arrest your users so that they will be savagely beaten in jail.

The technology problems that "secure" services are working on are all important. But we can't simply develop a good "secure" technology, consider it a niche product, and leave it at that. Those of us who are software development professionals need to build security into every product, because users expect it. Users expect it because we are, in a million implicit ways, telling them that they have it. If we put a "share with your friend!" button into a user interface, that's a claim: we're claiming that the data the user indicates is being shared only with their friend. Would we want to put in a button that says "share with your friend, and with us, and with the state security apparatus, and with any criminal who can break in and steal our database!"? Obviously not. So let's stop making the "share with your friend!" button actually do that.

Those of us who understand the importance of security and are in the business of creating secure software must, therefore, take on the Sisyphean task of not only creating good security, but of competing with the insecure software on its own turf, so that people actually use it. "Slightly worse to use than your regular email program, but secure" is not good enough. (Not to mention the fact that existing security solutions are more than "slightly" worse to use). Secure stuff has to be as good as or better than its insecure competitors.

I know that this is a monumental undertaking. I have personally tried and failed to do something like this more than once. As the Rabbi Tarfon put it, though:

It is not incumbent upon you to complete the work, but neither are you at liberty to desist from it.

25 Jan 2015 12:16am GMT

24 Jan 2015

feedPlanet Python

Holger Krekel: Thoughts on arguing end-to-end crypto and surveillance

Many Western governors are pushing for laws mandating all private communication can be secretly read and analyzed for them.The latest attack targets the one technology that still enables some privacy on a massively surveilled internet: end-to-end encryption. As hackers or IT people we can not afford to lament that the public doesn't understand the significance of end-to-end crypto or privacy if we don't appreciate its value for societies at home and abroad ourselves.

Responding to the renewed surveillance attacks with quick technical or narrow economic counter arguments is not going to work. An appropriate response needs to consider the political history and context of the current crypto and surveillance debates. Moreover, to stem the never-ending waves of new secret agency laws a re-framing of the common security debates is crucial to avoid the never-ending succession of new powers for government.

Let me start by rejecting the idea that governmental surveillance attacks have anything to do with fighting ruthless killers ("terrorists") however often this claim is repeated in broadcast media. This is not to disregard the power of repetition, see the endlessly repeated claims of the existence of "Weapons of mass destruction" as a pretext for the Iraq war, or the fact that advertisements work. But despite endless repetition, governmental surveillance attacks don't have anything to do with fighting terrorists. To turn it around, and i think the burden ought to be on the framers, where is the hard evidence that mass surveillance of civilians has significant effect, if any, on preventing terrorist attacks against civilians? And even if surveillance would prevent a few attacks how would it compare to the dangers of more government power?

The "fight terrorists with surveillance" discussion framing is seriously flawed also for another reason. Within it you are always going to lose the argument against more surveillance. If not now then after the next terror event. Because proponents can always argue they were right: if no attack happens it proves surveillance works and we need more of it. If an attack happens it also proves we need more surveillance. In this framed logic there can never be any rolling back of government powers.

The way out is to unframe the discussion and discuss the political and historical contexts of "terror attacks" and "expanding surveillance" separately. Let's start with surveillance. If fighting terrorism is a red herring what are the motivations and politics of expanding government surveillance?

Governors worry about their power base

Governors of all kinds worry that people decide to change things in ways which endanger the power their associated networks hold. And they are particularly afraid today because they know there are many reasons why people want to change things in more fundamental ways. As much as people have lost trust in governors, governors have lost trust into people to keep them and their ilk in power.

The fear of governors seems justified if you look at the example of Spain in 2015: big parts of Spain's social movements associate with a very new party on the block: Podemos. It aims to win the election in December and currently is leading the polls against the two parties which have governed Spain since 1975. It could actually happen despite the German chancellor Merkel supporting the Spanish president Rajoy who just introduced draconian laws against protesters and is generally sending his troops everywhere to avert the decline of his power network. Having to resort to direct repression is a sign of lost political power and in the case of Spain, panic. If you remember that Spain is a major EU country it's understandable that many other governors in the West are worried something similar might happen to them soon.

Governors are always afraid they could lose their sight and grip over what people in their constituency are up to. Today it is not enough to have buddies in broadcast media which frame the discussion and interpretation of events to the governor's liking. You also need to understand and contain, if possible, wider internet discussions before they can effect change you don't want. Governors learned from Hannah Arendt that private discussions form the basis for public opinions which in turn structure and determine governmental power. If that weren't the case how could feminist and really any social struggle have succeeded? It certainly never was the broadcast media or governors who first talked about and demanded rights for women or other oppressed groups.

How to contain decentralized communication?

New realities are co-created in a more decentralized manner and quicker than ever. Communication platforms grew in the last decade because of the interests of people to communicate and connect with one another. Maybe that's due to a lost sense of community in disintegrating city neighborhoods which make people use "social media". But in any case, Youtube, Twitter, Gmail, Facebook and IOS/Android app platforms became big because they facilitated decentralized communication and sharing between people. This presents a problem to governors because web communications are harder to contain in acceptable ways.

For a typical broadcast media discussion format you can send allied experts and construct "position" and "opposition" and thus frame the discussion. For example, it's acceptable to discuss the virtues and dangers of "press freedom", how to deal with "islamist militants" or how to "defend our values and rights". Western Governors find it much less acceptable to link the Hebdo killing of or the rise of the "Islamic State" to the recent Western wars in Iraq, Libya and Syria, or to the everyday killing of civilians through Western drones and torture. Governors can't yet directly contain such unacceptable linking activities and they are worried about it. For the time being, they try to frame it as irrelevant and repeat the "we are being attacked by ruthless killers" on broadcast media some more. It still kind of works but it's unclear for how long.

What helps to contain discussions is to implant "You are being watched!" into the minds of people discussing the future of their governance. Putting up some public examples of punishment for unacceptable dissent refines the message into "Watch your words (and internet links!)" … also known as internalized or self-censorship. That's not just effective for governors in Saudi Arabia but for their Western allies as well. The recent US sentencing of journalist Barret Brown to 48 months of prison for posting a link to some leaked data on an IRC channel can be seen as an example of a public punishment with chilling effects.

Arguments and national tactics against crypto attacks

Governors have long realized they can exploit central communication platform ownership to tap into most private communications. But to their apparent shock, many IT companies in the Post-Snowden era are implementing decentralized encryption because they in turn want to assure users that they can't surveil their private messages. As a reaction, governors are conspiring to prevent decentralized encryption reaching the masses which would see them losing their current in-depth access to private communication. Psychologically speaking, losing power is always harder to accept than not having it in the first place.

A response to the crypto attacks which I consider optimistic, if not shallow, is "it's not technically feasible to regulate or ban end-to-end crypto". It underestimates the ability of governors to write laws which will drastically change the playing field even if in an incremental manner. To begin with, why shouldn't it be possible to prevent companies from distributing apps which incorporate decentralized encryption? Google and Apple already employ their own regulation on what kind of apps are distributed through their stores. Another regulation on decentralized-crypto apps can probably be added by the governors in the US. And that would prevent decentralized encryption reaching the masses at least in the short term.

As to government access to end-to-end encryption, it's true that backdooring crypto would make people more vulnerable against all kind of exploiting attacks, not just governmental ones. Governors might frame this dillema by claiming that security against physical attacks is more important than security against someone reading your messages. Such an argument already incorporates the flawed "it's all about anti-terror" framing. The increased vulnerability of everyone's devices is a bit of a tricky issue for governors given they couldn't protect their own data against Snowden. If neccessary, governors will try to make concessions. Some applications such as online banking could be allowed to use non-backdoored crypto. They have all the banking data already, anyway. They probably will want to exempt governmental communication itself as well. With that we'd end up with a complete reversal of the democratic principle: public governments to act in secret and private communication to be constantly surveilled.

Western Governors have learned from the last Cryptowars battles. They know full well that they can only break private communication encryption if they outlaw it in a synchronized international manner. Otherwise they would have a harder time to overcome national arguments like "companies are going to leave the country if you ban decentral encryption". Therefore, we need to fend off attacks on decentralized crypto in at least some Western countries to make such commercial arguments useful. Concretely, US companies like Google and Apple will more strongly resist if the EU does not also illegalize decentralized crypto.

It is as crucial to prevent EU crypto regulations as it was two decades ago. During the crypto battles in the 1990ties I studied with the deeply inspiring Prof. Andreas Pfitzmann who consulted the German government on crypto regulation. Along with other colleagues and groups he tirelessly worked and finally turned the tides and prevented Germany and thus the EU from introducing government backdoors to crypto algorithms. This in turn lead France and then the US to drop their plans and eventually relax crypto export regulations to keep their companies competitive. Today, we are back to square zero and must again convince some EU governments or parliaments to refrain from crypto banning laws. It's a fight we better not lose.

Lastly, I'd like to be clear if maybe controversial on the dreadful Anti-Terror topic: If the Western governments want to stop killers from targetting western individuals they first need to stop ruthlessly killing and terrorizing individuals from abroad. Nothing else will bring more physical security against terrorist attacks. It reminds me of the 2500 year old question from the chinese politician and philosopher Confucius: "The way out is via the door. Why is it that no one will use this method?"


24 Jan 2015 11:40pm GMT

Holger Krekel: Thoughts on arguing end-to-end crypto and surveillance

Many Western governors are pushing for laws mandating all private communication can be secretly read and analyzed for them.The latest attack targets the one technology that still enables some privacy on a massively surveilled internet: end-to-end encryption. As hackers or IT people we can not afford to lament that the public doesn't understand the significance of end-to-end crypto or privacy if we don't appreciate its value for societies at home and abroad ourselves.

Responding to the renewed surveillance attacks with quick technical or narrow economic counter arguments is not going to work. An appropriate response needs to consider the political history and context of the current crypto and surveillance debates. Moreover, to stem the never-ending waves of new secret agency laws a re-framing of the common security debates is crucial to avoid the never-ending succession of new powers for government.

Let me start by rejecting the idea that governmental surveillance attacks have anything to do with fighting ruthless killers ("terrorists") however often this claim is repeated in broadcast media. This is not to disregard the power of repetition, see the endlessly repeated claims of the existence of "Weapons of mass destruction" as a pretext for the Iraq war, or the fact that advertisements work. But despite endless repetition, governmental surveillance attacks don't have anything to do with fighting terrorists. To turn it around, and i think the burden ought to be on the framers, where is the hard evidence that mass surveillance of civilians has significant effect, if any, on preventing terrorist attacks against civilians? And even if surveillance would prevent a few attacks how would it compare to the dangers of more government power?

The "fight terrorists with surveillance" discussion framing is seriously flawed also for another reason. Within it you are always going to lose the argument against more surveillance. If not now then after the next terror event. Because proponents can always argue they were right: if no attack happens it proves surveillance works and we need more of it. If an attack happens it also proves we need more surveillance. In this framed logic there can never be any rolling back of government powers.

The way out is to unframe the discussion and discuss the political and historical contexts of "terror attacks" and "expanding surveillance" separately. Let's start with surveillance. If fighting terrorism is a red herring what are the motivations and politics of expanding government surveillance?

Governors worry about their power base

Governors of all kinds worry that people decide to change things in ways which endanger the power their associated networks hold. And they are particularly afraid today because they know there are many reasons why people want to change things in more fundamental ways. As much as people have lost trust in governors, governors have lost trust into people to keep them and their ilk in power.

The fear of governors seems justified if you look at the example of Spain in 2015: big parts of Spain's social movements associate with a very new party on the block: Podemos. It aims to win the election in December and currently is leading the polls against the two parties which have governed Spain since 1975. It could actually happen despite the German chancellor Merkel supporting the Spanish president Rajoy who just introduced draconian laws against protesters and is generally sending his troops everywhere to avert the decline of his power network. Having to resort to direct repression is a sign of lost political power and in the case of Spain, panic. If you remember that Spain is a major EU country it's understandable that many other governors in the West are worried something similar might happen to them soon.

Governors are always afraid they could lose their sight and grip over what people in their constituency are up to. Today it is not enough to have buddies in broadcast media which frame the discussion and interpretation of events to the governor's liking. You also need to understand and contain, if possible, wider internet discussions before they can effect change you don't want. Governors learned from Hannah Arendt that private discussions form the basis for public opinions which in turn structure and determine governmental power. If that weren't the case how could feminist and really any social struggle have succeeded? It certainly never was the broadcast media or governors who first talked about and demanded rights for women or other oppressed groups.

How to contain decentralized communication?

New realities are co-created in a more decentralized manner and quicker than ever. Communication platforms grew in the last decade because of the interests of people to communicate and connect with one another. Maybe that's due to a lost sense of community in disintegrating city neighborhoods which make people use "social media". But in any case, Youtube, Twitter, Gmail, Facebook and IOS/Android app platforms became big because they facilitated decentralized communication and sharing between people. This presents a problem to governors because web communications are harder to contain in acceptable ways.

For a typical broadcast media discussion format you can send allied experts and construct "position" and "opposition" and thus frame the discussion. For example, it's acceptable to discuss the virtues and dangers of "press freedom", how to deal with "islamist militants" or how to "defend our values and rights". Western Governors find it much less acceptable to link the Hebdo killing of or the rise of the "Islamic State" to the recent Western wars in Iraq, Libya and Syria, or to the everyday killing of civilians through Western drones and torture. Governors can't yet directly contain such unacceptable linking activities and they are worried about it. For the time being, they try to frame it as irrelevant and repeat the "we are being attacked by ruthless killers" on broadcast media some more. It still kind of works but it's unclear for how long.

What helps to contain discussions is to implant "You are being watched!" into the minds of people discussing the future of their governance. Putting up some public examples of punishment for unacceptable dissent refines the message into "Watch your words (and internet links!)" … also known as internalized or self-censorship. That's not just effective for governors in Saudi Arabia but for their Western allies as well. The recent US sentencing of journalist Barret Brown to 48 months of prison for posting a link to some leaked data on an IRC channel can be seen as an example of a public punishment with chilling effects.

Arguments and national tactics against crypto attacks

Governors have long realized they can exploit central communication platform ownership to tap into most private communications. But to their apparent shock, many IT companies in the Post-Snowden era are implementing decentralized encryption because they in turn want to assure users that they can't surveil their private messages. As a reaction, governors are conspiring to prevent decentralized encryption reaching the masses which would see them losing their current in-depth access to private communication. Psychologically speaking, losing power is always harder to accept than not having it in the first place.

A response to the crypto attacks which I consider optimistic, if not shallow, is "it's not technically feasible to regulate or ban end-to-end crypto". It underestimates the ability of governors to write laws which will drastically change the playing field even if in an incremental manner. To begin with, why shouldn't it be possible to prevent companies from distributing apps which incorporate decentralized encryption? Google and Apple already employ their own regulation on what kind of apps are distributed through their stores. Another regulation on decentralized-crypto apps can probably be added by the governors in the US. And that would prevent decentralized encryption reaching the masses at least in the short term.

As to government access to end-to-end encryption, it's true that backdooring crypto would make people more vulnerable against all kind of exploiting attacks, not just governmental ones. Governors might frame this dillema by claiming that security against physical attacks is more important than security against someone reading your messages. Such an argument already incorporates the flawed "it's all about anti-terror" framing. The increased vulnerability of everyone's devices is a bit of a tricky issue for governors given they couldn't protect their own data against Snowden. If neccessary, governors will try to make concessions. Some applications such as online banking could be allowed to use non-backdoored crypto. They have all the banking data already, anyway. They probably will want to exempt governmental communication itself as well. With that we'd end up with a complete reversal of the democratic principle: public governments to act in secret and private communication to be constantly surveilled.

Western Governors have learned from the last Cryptowars battles. They know full well that they can only break private communication encryption if they outlaw it in a synchronized international manner. Otherwise they would have a harder time to overcome national arguments like "companies are going to leave the country if you ban decentral encryption". Therefore, we need to fend off attacks on decentralized crypto in at least some Western countries to make such commercial arguments useful. Concretely, US companies like Google and Apple will more strongly resist if the EU does not also illegalize decentralized crypto.

It is as crucial to prevent EU crypto regulations as it was two decades ago. During the crypto battles in the 1990ties I studied with the deeply inspiring Prof. Andreas Pfitzmann who consulted the German government on crypto regulation. Along with other colleagues and groups he tirelessly worked and finally turned the tides and prevented Germany and thus the EU from introducing government backdoors to crypto algorithms. This in turn lead France and then the US to drop their plans and eventually relax crypto export regulations to keep their companies competitive. Today, we are back to square zero and must again convince some EU governments or parliaments to refrain from crypto banning laws. It's a fight we better not lose.

Lastly, I'd like to be clear if maybe controversial on the dreadful Anti-Terror topic: If the Western governments want to stop killers from targetting western individuals they first need to stop ruthlessly killing and terrorizing individuals from abroad. Nothing else will bring more physical security against terrorist attacks. It reminds me of the 2500 year old question from the chinese politician and philosopher Confucius: "The way out is via the door. Why is it that no one will use this method?"


24 Jan 2015 11:40pm GMT

23 Jan 2015

feedPlanet Python

Python Software Foundation: 2014 Year in Review, Part 2

Today's post wraps up our brief review of 2014, with a summary of both technical and community developments.
On the technical side, the Python language grew with the releases of Python 2.7.9, 3.3.5, 3.4, and, in August, 3.4.1. Major new features of the 3.4 series, compared to 3.3 include "hundreds of small improvements and bug fixes." Additionally, Python 3.4.1 has many more advantages. "One notable change: the version of OpenSSL bundled with the Windows installer no longer has the "HeartBleed" vulnerability." See python.org for more detailed information and to download any of these latest versions (as well as a Python 3 compatible version of PyPy).
The PSF also saw a culmination of a lot of hard work in the release of our new website, pydotorg. This site serves as a main repository of all crucial information about the Python language (downloads of all versions, tutorials, documentation, new releases, ongoing developments, ways to contribute), the PSF (what we are, what we do, bylaws, membership info, meeting minutes and resolutions), community resources (Python Wiki, the IRC, the diversity policy, mailing list, merchandise, projects, and events). Visit the website at python.org to see how it should be your first stop for anything Python related and to learn how you can contribute to its upkeep.
One of the PSF's favorite activities is to honor the contributions of its members. The 2014 Community Service Awards went to Pythonistas whose work, from organizing the largest annual PyCon, to teaching future Python users, to developing important modules and libraries that enhance the usefulness of Python, benefited so many of us. Congratulations to the following recipients:
Diana Clarke "for her work with the Canadian Python community, her organizing efforts for PyCon CA and PyCon US over the past several years, and her mentorship of many others in the community;"
R. David Murray "for his work as a core committer and as a long-time mentor of new contributors;"
Barbara Shaurette and Katie Cunningham "in recognition of their work to create and run their Young Coders classes, along with freely distributing their teaching materials;"
Christophe Gohke, of the University of California, Irvine; and Armin Ronacher, founding member of the Pocoo team; for their technical contributions to Python.
PyCon 2014 in April was the largest ever, and the first held outside the U.S. The beautiful and accommodating Palais de Congres in Montreal allowed for an incredibly smooth, comfortable, and well-organized week (in addition, of course, to the efforts of Diana Clarke and numerous volunteers). There were over 2,500 attendees, 128 sponsors, and 95 talks (selected from over 300 submissions) over 5 simultaneously-running tracks. In addition to the usual tutorials, lightening talks, and sprints, the conference offered first-time childcare, a service that enabled huge participation in a Young-Coders' class, as well as a hectic and productive Education Track, and a lot of youthful energy and enthusiasm. The city of Montreal itself provided for a "sixth track," Explore Montreal, allowing attendees to tour Old Montreal," visit museums, and climb "Mount Royal." If you missed last year's PyCon, it's not too late to register for PyCon 2015 to visit this fabulous city and attend an amazing conference.
Other activities performed by the PSF in 2014 included developing a new marketing brochure, choosing Portland, OR as the site for PyCon 2016 and 2017, updating the bylaws regarding memberships, quorums, and the grants procedure, adopting a privacy policy, and keeping up with trademark registrations for the term PyCon.

23 Jan 2015 7:44pm GMT

Python Software Foundation: 2014 Year in Review, Part 2

Today's post wraps up our brief review of 2014, with a summary of both technical and community developments.
On the technical side, the Python language grew with the releases of Python 2.7.9, 3.3.5, 3.4, and, in August, 3.4.1. Major new features of the 3.4 series, compared to 3.3 include "hundreds of small improvements and bug fixes." Additionally, Python 3.4.1 has many more advantages. "One notable change: the version of OpenSSL bundled with the Windows installer no longer has the "HeartBleed" vulnerability." See python.org for more detailed information and to download any of these latest versions (as well as a Python 3 compatible version of PyPy).
The PSF also saw a culmination of a lot of hard work in the release of our new website, pydotorg. This site serves as a main repository of all crucial information about the Python language (downloads of all versions, tutorials, documentation, new releases, ongoing developments, ways to contribute), the PSF (what we are, what we do, bylaws, membership info, meeting minutes and resolutions), community resources (Python Wiki, the IRC, the diversity policy, mailing list, merchandise, projects, and events). Visit the website at python.org to see how it should be your first stop for anything Python related and to learn how you can contribute to its upkeep.
One of the PSF's favorite activities is to honor the contributions of its members. The 2014 Community Service Awards went to Pythonistas whose work, from organizing the largest annual PyCon, to teaching future Python users, to developing important modules and libraries that enhance the usefulness of Python, benefited so many of us. Congratulations to the following recipients:
Diana Clarke "for her work with the Canadian Python community, her organizing efforts for PyCon CA and PyCon US over the past several years, and her mentorship of many others in the community;"
R. David Murray "for his work as a core committer and as a long-time mentor of new contributors;"
Barbara Shaurette and Katie Cunningham "in recognition of their work to create and run their Young Coders classes, along with freely distributing their teaching materials;"
Christophe Gohke, of the University of California, Irvine; and Armin Ronacher, founding member of the Pocoo team; for their technical contributions to Python.
PyCon 2014 in April was the largest ever, and the first held outside the U.S. The beautiful and accommodating Palais de Congres in Montreal allowed for an incredibly smooth, comfortable, and well-organized week (in addition, of course, to the efforts of Diana Clarke and numerous volunteers). There were over 2,500 attendees, 128 sponsors, and 95 talks (selected from over 300 submissions) over 5 simultaneously-running tracks. In addition to the usual tutorials, lightening talks, and sprints, the conference offered first-time childcare, a service that enabled huge participation in a Young-Coders' class, as well as a hectic and productive Education Track, and a lot of youthful energy and enthusiasm. The city of Montreal itself provided for a "sixth track," Explore Montreal, allowing attendees to tour Old Montreal," visit museums, and climb "Mount Royal." If you missed last year's PyCon, it's not too late to register for PyCon 2015 to visit this fabulous city and attend an amazing conference.
Other activities performed by the PSF in 2014 included developing a new marketing brochure, choosing Portland, OR as the site for PyCon 2016 and 2017, updating the bylaws regarding memberships, quorums, and the grants procedure, adopting a privacy policy, and keeping up with trademark registrations for the term PyCon.

23 Jan 2015 7:44pm GMT

Brett Cannon: Why WhatsApp made its web app use your phone as a server

Ever since WhatsApp announced their web app, I have seen various people complain about having to keep your phone on to send messages. But in all of these posts people seem to be overlooking two key points about the design of WhatsApp which either necessitate this design or at least facilitate WhatsApp in keeping their service lean and fast.

Encryption

In case you didn't hear, WhatsApp (supposedly) does end-to-end encryption. That means WhatsApp can't read what people say in their messages. What this also means, though, is that if someone sends you a message and you have your phone and web app, there is no way to read that message independently of both devices without sharing the decryption key, which is not good security practice. You could have separate keys for your phone and web browser, but then you would then need to make sure to share keys between your phone, your browser, and all of your contacts so they could receive messages from any of your devices.

But if you treat your phone as your personal WhatsApp server, then your phone can continue to have the master keys for your account and then you only need to manage keys between your phone and your other devices, making it a hub-and-spoke system where you phone is the hub and your other devices are the spokes. This keeps WhatsApp out of the key management business and allows you to easily revoke keys for your other devices without WhatsApp having to store anything for you. So tying everything through your phone keeps everything encrypted and simplifies key sharing between users to just their phones and keeps security simple which is how you want it to be.

Storage

Someone pointed out to me that WhatsApp is not in the storage business, and when you stop and look at how the service was structured before the web app came along you will realize that WhatsApp tried to minimize what data it had to store from its inception. For instance, if you get a new phone you actually have to migrate your messages over yourself. Since WhatsApp doesn't keep messages on their servers, your phone ends up being the keeper of truth when it comes to what messages you sent and received.

But the key thing about WhatsApp not storing messages on their servers is how much it simplifies their service. Consider their last publicly stated user count of 500,000,000. Since WhatsApp doesn't store messages for you, they really only need to store messages that have yet to be delivered to a user's phone and your account's configuration data. So let's assume every user suddenly sent a bunch of photos that came to a total of 1 MB of pending messages (remember that WhatsApp is only going to show you a small version of a photo and so they can compress them such that they don't take up much space; 1 MB should go a long way). That's 1 MB * 500,000,000 = 500 TB of storage. OK, not a puny number for most services.

But let's look at this from a cloud perspective. Let's say you wanted an extremely fast service, so you would want to use local SSD which Google Compute Engine offers. As I write this, a local SSD on GCE is 375 GB and you can have up to 4 per instance. At 375 GB that gets means you would need 1,334 SSDs to store 500 TB of data (it would also require at minimum 334 instances and probably enough for the service to run, but I'm not pricing out computation or bandwidth costs to keep this simple). Now according to GCE's pricing, a local SSD costs USD 81.75/month. That means it would cost you USD 81.75 * 1,334 SSDs = USD 109,054.50/month or USD 1,308,654/year. Now let's be totally extravagant and say you want the data replicated near the sender, near the receiver, and on one other continent for safekeeping until delivery (when the sender and receiver are on the same continent the data could go to two separate clusters). So we are talking about 3N data replication. That works out to USD 3,925,962/year in local SSD storage costs if you used Google Compute Engine at full retail and always maintained this level of storage constantly. In the current startup climate, USD 4 million/year is pittance (and unnecessary as I bet WhatsApp could get away with 2N data replication if that since this is only for buffering purposes). And for a company like Facebook who have their own clusters? Storing 1.5 PB of data would not be difficult at all, so WhatsApp could go as far as backing up the data on every populated continent if they wanted to for 3 PB or less than USD 8 million/year.

In other words by relying on your phone as the storage mechanism for messages and not having to keep anything on their own servers, WhatsApp can run very cheaply and efficiently (and only cost you USD 1/year as a user). This makes WhatsApp basically nothing more than a fast routing service with some buffering between phones, much like the telcos (which might be what inspired WhatsApp to use Erlang). It's rather ingenious and probably why the service seems so fast and has such great uptime.

And I bet users don't care about the potential loss of messages either; when was the last time you scrolled back into the history on your phone to a point that preceded you getting that phone or having catastrophic data loss on the device? Plus you have to consider how many users of WhatsApp are really going to use the web app; I bet a large portion of their users don't even have their own computers beyond their phones so this web app service probably isn't critical to WhatsApp's success.

23 Jan 2015 5:27pm GMT

Brett Cannon: Why WhatsApp made its web app use your phone as a server

Ever since WhatsApp announced their web app, I have seen various people complain about having to keep your phone on to send messages. But in all of these posts people seem to be overlooking two key points about the design of WhatsApp which either necessitate this design or at least facilitate WhatsApp in keeping their service lean and fast.

Encryption

In case you didn't hear, WhatsApp (supposedly) does end-to-end encryption. That means WhatsApp can't read what people say in their messages. What this also means, though, is that if someone sends you a message and you have your phone and web app, there is no way to read that message independently of both devices without sharing the decryption key, which is not good security practice. You could have separate keys for your phone and web browser, but then you would then need to make sure to share keys between your phone, your browser, and all of your contacts so they could receive messages from any of your devices.

But if you treat your phone as your personal WhatsApp server, then your phone can continue to have the master keys for your account and then you only need to manage keys between your phone and your other devices, making it a hub-and-spoke system where you phone is the hub and your other devices are the spokes. This keeps WhatsApp out of the key management business and allows you to easily revoke keys for your other devices without WhatsApp having to store anything for you. So tying everything through your phone keeps everything encrypted and simplifies key sharing between users to just their phones and keeps security simple which is how you want it to be.

Storage

Someone pointed out to me that WhatsApp is not in the storage business, and when you stop and look at how the service was structured before the web app came along you will realize that WhatsApp tried to minimize what data it had to store from its inception. For instance, if you get a new phone you actually have to migrate your messages over yourself. Since WhatsApp doesn't keep messages on their servers, your phone ends up being the keeper of truth when it comes to what messages you sent and received.

But the key thing about WhatsApp not storing messages on their servers is how much it simplifies their service. Consider their last publicly stated user count of 500,000,000. Since WhatsApp doesn't store messages for you, they really only need to store messages that have yet to be delivered to a user's phone and your account's configuration data. So let's assume every user suddenly sent a bunch of photos that came to a total of 1 MB of pending messages (remember that WhatsApp is only going to show you a small version of a photo and so they can compress them such that they don't take up much space; 1 MB should go a long way). That's 1 MB * 500,000,000 = 500 TB of storage. OK, not a puny number for most services.

But let's look at this from a cloud perspective. Let's say you wanted an extremely fast service, so you would want to use local SSD which Google Compute Engine offers. As I write this, a local SSD on GCE is 375 GB and you can have up to 4 per instance. At 375 GB that gets means you would need 1,334 SSDs to store 500 TB of data (it would also require at minimum 334 instances and probably enough for the service to run, but I'm not pricing out computation or bandwidth costs to keep this simple). Now according to GCE's pricing, a local SSD costs USD 81.75/month. That means it would cost you USD 81.75 * 1,334 SSDs = USD 109,054.50/month or USD 1,308,654/year. Now let's be totally extravagant and say you want the data replicated near the sender, near the receiver, and on one other continent for safekeeping until delivery (when the sender and receiver are on the same continent the data could go to two separate clusters). So we are talking about 3N data replication. That works out to USD 3,925,962/year in local SSD storage costs if you used Google Compute Engine at full retail and always maintained this level of storage constantly. In the current startup climate, USD 4 million/year is pittance (and unnecessary as I bet WhatsApp could get away with 2N data replication if that since this is only for buffering purposes). And for a company like Facebook who have their own clusters? Storing 1.5 PB of data would not be difficult at all, so WhatsApp could go as far as backing up the data on every populated continent if they wanted to for 3 PB or less than USD 8 million/year.

In other words by relying on your phone as the storage mechanism for messages and not having to keep anything on their own servers, WhatsApp can run very cheaply and efficiently (and only cost you USD 1/year as a user). This makes WhatsApp basically nothing more than a fast routing service with some buffering between phones, much like the telcos (which might be what inspired WhatsApp to use Erlang). It's rather ingenious and probably why the service seems so fast and has such great uptime.

And I bet users don't care about the potential loss of messages either; when was the last time you scrolled back into the history on your phone to a point that preceded you getting that phone or having catastrophic data loss on the device? Plus you have to consider how many users of WhatsApp are really going to use the web app; I bet a large portion of their users don't even have their own computers beyond their phones so this web app service probably isn't critical to WhatsApp's success.

23 Jan 2015 5:27pm GMT

Caktus Consulting Group: Why I Love Technical Blogging

I love writing blog posts, and today I'm setting out to do something I've never tried before: write a blog post about writing blog posts. A big part of our mission at Caktus is to foster and help grow the Python and Django development communities, both locally and nationally. Part of how we've tried to accomplish this in the past is through hosting development sprints, sponsoring and attending conferences such as PyCon and DjangoCon, and building a knowledge base of common problems in Python and Django development in our blog. Many in the Django community first get to know Caktus through our blog, and it's both gratifying and humbling when I meet someone at a conference and the person thanks me for a post Caktus wrote that helped him or her solve a technical problem at some point in the past.

While I personally don't do as much software development as I used to and hence no longer write as many technical posts, the Caktus blog and many others in the community continue as a constant source of inspiration and education to me. As software developers we are constantly trying to work ourselves out of a job, building tools that organize information and help people communicate. Sharing a brief, highly specific technical blog post serves in a similar capacity; after I've spent 1-2 hours or more researching something that ultimately took 5-10 minutes to fix, I'd hate for someone else to have to go through the same experience. Writing up a quick, 1-2 paragraph technical post about the issue not only helps me think through the problem, but also hopefully saves a few minutes of someone else's life at some point in the future.

To help me better understand what I like so much about blogging, I went back and reviewed the history of Caktus blogging efforts over the past 5 years and separated our posts into categories. While I'm sure there are innumerable ways to do this, in case it serves as a source of inspiration to others, what follows are the categories I came up with:

A big thank you to everyone in the Python and Django community for being open and willing to share your experiences and problem solving efforts. Without this, Caktus would not be where it is today and for that I am deeply grateful. If this post happens to inspire at least one short technical post from someone who hasn't written one before, I'll consider it a success.

23 Jan 2015 4:06pm GMT

Caktus Consulting Group: Why I Love Technical Blogging

I love writing blog posts, and today I'm setting out to do something I've never tried before: write a blog post about writing blog posts. A big part of our mission at Caktus is to foster and help grow the Python and Django development communities, both locally and nationally. Part of how we've tried to accomplish this in the past is through hosting development sprints, sponsoring and attending conferences such as PyCon and DjangoCon, and building a knowledge base of common problems in Python and Django development in our blog. Many in the Django community first get to know Caktus through our blog, and it's both gratifying and humbling when I meet someone at a conference and the person thanks me for a post Caktus wrote that helped him or her solve a technical problem at some point in the past.

While I personally don't do as much software development as I used to and hence no longer write as many technical posts, the Caktus blog and many others in the community continue as a constant source of inspiration and education to me. As software developers we are constantly trying to work ourselves out of a job, building tools that organize information and help people communicate. Sharing a brief, highly specific technical blog post serves in a similar capacity; after I've spent 1-2 hours or more researching something that ultimately took 5-10 minutes to fix, I'd hate for someone else to have to go through the same experience. Writing up a quick, 1-2 paragraph technical post about the issue not only helps me think through the problem, but also hopefully saves a few minutes of someone else's life at some point in the future.

To help me better understand what I like so much about blogging, I went back and reviewed the history of Caktus blogging efforts over the past 5 years and separated our posts into categories. While I'm sure there are innumerable ways to do this, in case it serves as a source of inspiration to others, what follows are the categories I came up with:

A big thank you to everyone in the Python and Django community for being open and willing to share your experiences and problem solving efforts. Without this, Caktus would not be where it is today and for that I am deeply grateful. If this post happens to inspire at least one short technical post from someone who hasn't written one before, I'll consider it a success.

23 Jan 2015 4:06pm GMT

Vasudev Ram: PrettyTable to PDF is pretty easy with xtopdf

By Vasudev Ram


"PrettyTable to PDF is pretty easy with xtopdf."

How's that for some alliteration? :)

PrettyTable is a Python library to help you generate nice tables with ASCII characters as the borders, plus alignment of text within columns, headings, padding, etc.

Excerpt from the site:

[ PrettyTable is a simple Python library designed to make it quick and easy to represent tabular data in visually appealing ASCII tables.
...
PrettyTable lets you control many aspects of the table, like the width of the column padding, the alignment of text within columns, which characters are used to draw the table border, whether you even want a border, and much more. You can control which subsets of the columns and rows are printed, and you can sort the rows by the value of a particular column.

PrettyTable can also generate HTML code with the data in a <table> structure. ]

I came across PrettyTable via this blog post:

11 Python Libraries You Might Not Know.

Then I thought of using it with my PDF creation toolkit, to generate such ASCII tables, but as PDF. Here's a program, PrettyTableToPDF.py, that shows how to do that:

"""
PrettyTableToPDF.py
A demo program to show how to convert the output generated
by the PrettyTable library, to PDF, using the xtopdf toolkit
for PDF creation from other formats.
Author: Vasudev Ram - http://www.dancingbison.com
xtopdf is at: http://slides.com/vasudevram/xtopdf

Copyright 2015 Vasudev Ram
"""

from prettytable import PrettyTable
from PDFWriter import PDFWriter

pt = PrettyTable(["City name", "Area", "Population", "Annual Rainfall"])
pt.align["City name"] = "l" # Left align city names
pt.padding_width = 1 # One space between column edges and contents (default)
pt.add_row(["Adelaide",1295, 1158259, 600.5])
pt.add_row(["Brisbane",5905, 1857594, 1146.4])
pt.add_row(["Darwin", 112, 120900, 1714.7])
pt.add_row(["Hobart", 1357, 205556, 619.5])
pt.add_row(["Sydney", 2058, 4336374, 1214.8])
pt.add_row(["Melbourne", 1566, 3806092, 646.9])
pt.add_row(["Perth", 5386, 1554769, 869.4])
lines = pt.get_string()

pw = PDFWriter('Australia-Rainfall.pdf')
pw.setFont('Courier', 12)
pw.setHeader('Demo of PrettyTable to PDF')
pw.setFooter('Demo of PrettyTable to PDF')
for line in lines.split('\n'):
pw.writeLine(line)
pw.close()


You can run the program with:

$ python PrettyTableToPDF.py

And here is a screenshot of the output PDF in Foxit PDF Reader:


- Enjoy.

--- Posts about Python --- Posts about xtopdf ---

- Vasudev Ram - Dancing Bison Enterprises - Python programming and trainingSignup to be informed about my new products or services.Contact Page

Share |
Vasudev Ram

23 Jan 2015 8:52am GMT

Vasudev Ram: PrettyTable to PDF is pretty easy with xtopdf

By Vasudev Ram


"PrettyTable to PDF is pretty easy with xtopdf."

How's that for some alliteration? :)

PrettyTable is a Python library to help you generate nice tables with ASCII characters as the borders, plus alignment of text within columns, headings, padding, etc.

Excerpt from the site:

[ PrettyTable is a simple Python library designed to make it quick and easy to represent tabular data in visually appealing ASCII tables.
...
PrettyTable lets you control many aspects of the table, like the width of the column padding, the alignment of text within columns, which characters are used to draw the table border, whether you even want a border, and much more. You can control which subsets of the columns and rows are printed, and you can sort the rows by the value of a particular column.

PrettyTable can also generate HTML code with the data in a <table> structure. ]

I came across PrettyTable via this blog post:

11 Python Libraries You Might Not Know.

Then I thought of using it with my PDF creation toolkit, to generate such ASCII tables, but as PDF. Here's a program, PrettyTableToPDF.py, that shows how to do that:

"""
PrettyTableToPDF.py
A demo program to show how to convert the output generated
by the PrettyTable library, to PDF, using the xtopdf toolkit
for PDF creation from other formats.
Author: Vasudev Ram - http://www.dancingbison.com
xtopdf is at: http://slides.com/vasudevram/xtopdf

Copyright 2015 Vasudev Ram
"""

from prettytable import PrettyTable
from PDFWriter import PDFWriter

pt = PrettyTable(["City name", "Area", "Population", "Annual Rainfall"])
pt.align["City name"] = "l" # Left align city names
pt.padding_width = 1 # One space between column edges and contents (default)
pt.add_row(["Adelaide",1295, 1158259, 600.5])
pt.add_row(["Brisbane",5905, 1857594, 1146.4])
pt.add_row(["Darwin", 112, 120900, 1714.7])
pt.add_row(["Hobart", 1357, 205556, 619.5])
pt.add_row(["Sydney", 2058, 4336374, 1214.8])
pt.add_row(["Melbourne", 1566, 3806092, 646.9])
pt.add_row(["Perth", 5386, 1554769, 869.4])
lines = pt.get_string()

pw = PDFWriter('Australia-Rainfall.pdf')
pw.setFont('Courier', 12)
pw.setHeader('Demo of PrettyTable to PDF')
pw.setFooter('Demo of PrettyTable to PDF')
for line in lines.split('\n'):
pw.writeLine(line)
pw.close()


You can run the program with:

$ python PrettyTableToPDF.py

And here is a screenshot of the output PDF in Foxit PDF Reader:


- Enjoy.

--- Posts about Python --- Posts about xtopdf ---

- Vasudev Ram - Dancing Bison Enterprises - Python programming and trainingSignup to be informed about my new products or services.Contact Page

Share |
Vasudev Ram

23 Jan 2015 8:52am GMT

Greg Taylor: Why you should donate to the Django fellowship program

Disclaimer: I do not represent the Django Software Foundation in any way, nor has anything below been endorsed by the DSF. The following opinions are my own, unsolicited rambling.

If you hadn't been looking for it specifically, you may have missed it. The Django Softare Foundation is running a fundraising effort for the new Django Fellowship program. It sounds like they're still trying to figure out how to get the word out, so I wanted to do what I could to tell you why you should chip in.

This particular blog post is going to focus on encouraging (peer-pressuring) commercial Django users in particular, though enthusiasts are welcome to read along!

Humble beginnings

Django is free and open source. Just provide the expertise and the infrastructure and you can build just about whatever web powered contraption you'd like. So you end up doing just that.

Your first stop is the Django tutorial, written and maintained by a community of volunteers (just like the rest framework itself). You stumble along, slowly at first. Perhaps you find yourself frustrated at times, or maybe things move along at a faster pace. In no time, you've got "Hello World!" rendering, and here comes a business idea!

One hundred lines of code turns into a thousand, then five thousand, and beyond. You start seeing signups, and revenue begins to trickle in. You toil away at your codebase, making improvements and dealing with the "accidental features" that crept in during one of your late night dev sessions.

You could have built your business on one of any number of frameworks, but you chose Django. You like how it's a very productive way to build a web app. You appreciate how it's not impossible to find Django developers to work with you. There are probably some things you don't like, but you might not have the time to work on fixing them yourself. You're just busy shipping and growing.

But it could be better still!

You're happily using Django, it serves you well. There are a few things you'd love to see fixed or improved, but you don't really have the time or expertise to contribute directly. As luck would have it, all of the Django core developers have day jobs themselves. Things would progress much more quickly if we had someone working full-time on Django…

Enter: Django Fellowship Program. The idea is to fund at least one Django developer to work for the DSF part or full-time for a while. During this fellowship, said developer sets aside some or all of their other responsibilities to focus on improving Django. The DSF, in turn, pays the developer a fair (but low rate) for their work.

As per the Tim Graham's recent retrospective blog post, we've see some huge leaps forward for the project during these fellowships. These are periods of focus and rapid improvement that everyone (including your business) benefit from.

The only problem is that we're not going to see the benefits of this program unless it gets (and stays) funded. A well-funded fellowship program could mean one (or more) developers working on Django full-time at any given point in time. That would be huge for the project (and you and I).

Why you should donate

As a business, we are donating to the fellowship program to see one of our critical components improved. Due to the fellowship application process, you can be assured that your money will be paying a capable, trusted developer to get things done.

Consequently, you can view a donation to the Django Fellowship program as an investment with an almost assuredly positive return. If you are making money with Django, consider making a (potentially tax-deductible) investment in what may be the foundation of your business.

At the end of the first full day of fund-raising, there are precious few commercial donors listed in the "Django Heroes" leaderboard. Let's help change that!

If you don't hold the purse strings at your business, get in touch with someone who does and tell them about this investment with near-guaranteed returns.

23 Jan 2015 7:21am GMT

Greg Taylor: Why you should donate to the Django fellowship program

Disclaimer: I do not represent the Django Software Foundation in any way, nor has anything below been endorsed by the DSF. The following opinions are my own, unsolicited rambling.

If you hadn't been looking for it specifically, you may have missed it. The Django Softare Foundation is running a fundraising effort for the new Django Fellowship program. It sounds like they're still trying to figure out how to get the word out, so I wanted to do what I could to tell you why you should chip in.

This particular blog post is going to focus on encouraging (peer-pressuring) commercial Django users in particular, though enthusiasts are welcome to read along!

Humble beginnings

Django is free and open source. Just provide the expertise and the infrastructure and you can build just about whatever web powered contraption you'd like. So you end up doing just that.

Your first stop is the Django tutorial, written and maintained by a community of volunteers (just like the rest framework itself). You stumble along, slowly at first. Perhaps you find yourself frustrated at times, or maybe things move along at a faster pace. In no time, you've got "Hello World!" rendering, and here comes a business idea!

One hundred lines of code turns into a thousand, then five thousand, and beyond. You start seeing signups, and revenue begins to trickle in. You toil away at your codebase, making improvements and dealing with the "accidental features" that crept in during one of your late night dev sessions.

You could have built your business on one of any number of frameworks, but you chose Django. You like how it's a very productive way to build a web app. You appreciate how it's not impossible to find Django developers to work with you. There are probably some things you don't like, but you might not have the time to work on fixing them yourself. You're just busy shipping and growing.

But it could be better still!

You're happily using Django, it serves you well. There are a few things you'd love to see fixed or improved, but you don't really have the time or expertise to contribute directly. As luck would have it, all of the Django core developers have day jobs themselves. Things would progress much more quickly if we had someone working full-time on Django…

Enter: Django Fellowship Program. The idea is to fund at least one Django developer to work for the DSF part or full-time for a while. During this fellowship, said developer sets aside some or all of their other responsibilities to focus on improving Django. The DSF, in turn, pays the developer a fair (but low rate) for their work.

As per the Tim Graham's recent retrospective blog post, we've see some huge leaps forward for the project during these fellowships. These are periods of focus and rapid improvement that everyone (including your business) benefit from.

The only problem is that we're not going to see the benefits of this program unless it gets (and stays) funded. A well-funded fellowship program could mean one (or more) developers working on Django full-time at any given point in time. That would be huge for the project (and you and I).

Why you should donate

As a business, we are donating to the fellowship program to see one of our critical components improved. Due to the fellowship application process, you can be assured that your money will be paying a capable, trusted developer to get things done.

Consequently, you can view a donation to the Django Fellowship program as an investment with an almost assuredly positive return. If you are making money with Django, consider making a (potentially tax-deductible) investment in what may be the foundation of your business.

At the end of the first full day of fund-raising, there are precious few commercial donors listed in the "Django Heroes" leaderboard. Let's help change that!

If you don't hold the purse strings at your business, get in touch with someone who does and tell them about this investment with near-guaranteed returns.

23 Jan 2015 7:21am GMT

Greg Taylor: Let’s play: python-gotalk

A recent HackerNews post announced Gotalk, a simple bidirectional protocol. I can imagine your collective eyeballs rolling. "Oh great, yet another half-baked way for… things to talk to one other". But keep following along, maybe you'll see something you like. Here are some highlights:

The official Gotalk repo has a bunch of examples and even some helper libraries in Go and JS. The official implementation is in Go, with the JS libraries being built on top of WebSockets.

But what about Python?!?

"But Greg, I too want to play with this fledging version 0.0 protocol! And I want to do it in Python!"

You, my friend, are in luck! I have a rough, version 0.0 Python module to go with this new, version 0.0 protocol.

For now, most of the effort is on message serialization and deserialization. We'll be keeping that separate from any of the naughty bits (sockets, IO, and other things). The goal is to provide some reusable components that people can tinker with.

And more importantly, I haven't ever bothered to mess around at the protocol level very often. This has been a great excuse to play around with a spec that is just getting started.

Pull requests, issues, suggestions, and the whole lot are welcome. Tell me how bad I screwed up!

Closing notes

If it wasn't already evident, do not use this for anything but tinkering right now!.

Also, to stem the tide of "Well, why didn't you just use X instead?", this is a fun little experiment for me. Yes, I wrote a protocol serialization/deserialization for funzies. No, I'm not mentally ill.

23 Jan 2015 5:55am GMT

Greg Taylor: Let’s play: python-gotalk

A recent HackerNews post announced Gotalk, a simple bidirectional protocol. I can imagine your collective eyeballs rolling. "Oh great, yet another half-baked way for… things to talk to one other". But keep following along, maybe you'll see something you like. Here are some highlights:

The official Gotalk repo has a bunch of examples and even some helper libraries in Go and JS. The official implementation is in Go, with the JS libraries being built on top of WebSockets.

But what about Python?!?

"But Greg, I too want to play with this fledging version 0.0 protocol! And I want to do it in Python!"

You, my friend, are in luck! I have a rough, version 0.0 Python module to go with this new, version 0.0 protocol.

For now, most of the effort is on message serialization and deserialization. We'll be keeping that separate from any of the naughty bits (sockets, IO, and other things). The goal is to provide some reusable components that people can tinker with.

And more importantly, I haven't ever bothered to mess around at the protocol level very often. This has been a great excuse to play around with a spec that is just getting started.

Pull requests, issues, suggestions, and the whole lot are welcome. Tell me how bad I screwed up!

Closing notes

If it wasn't already evident, do not use this for anything but tinkering right now!.

Also, to stem the tide of "Well, why didn't you just use X instead?", this is a fun little experiment for me. Yes, I wrote a protocol serialization/deserialization for funzies. No, I'm not mentally ill.

23 Jan 2015 5:55am GMT

Mike C. Fletcher: Type Declarations for Python 2.7

So there's a big flamey thread over on python-list about the Python 3.x type-annotation. Thing is, I can't say the annotations seem all *that* important to the behaviour (static type checking): Consider this Python 2.7 decorator-set:

def _func_annotation_for(func):
    """Retrieve the function annotation for a given function or create it"""
    current = getattr(func,'func_annotation',None)
    if current is None:
        current = func.func_annotation = {}
    return current 
def types( *types, **named ):
    def annotate(function):
        base = _func_annotation_for(function)
        base.update(named)
        if hasattr( function, 'func_code' ): # python 2.x
            names = function.func_code.co_varnames
        else:
            names = function.__code__.co_varnames
        
        for name, typ in zip(names, types):
            base[name] = typ
        return function 
    return annotate
def returns( typ ):
    def annotate(function):
        _func_annotation_for(function)['return'] = typ 
        return function 
    return annotate

That's a few lines of code and would seem to do the bulk of what type annotations do, namely populate a func_annotation dictionary and provide a textual clue as to where the type annotations are for AST or similar tools. It's true that it doesn't look as integrated as the syntax-based annotations, but to my eyes it actually looks cleaner when reading:

@types((int, float, complex), d=int, b=str)
@returns(float)
def x(a, b, c, d=3):
    return float(d)

The extra fluff (type annotations) is separated out and doesn't cloud the parameter list, you can use long type expressions to specify unions-of-types and the like without making it impossible to see your parameters. You can just annotate the functions/parameters you care about... etc. Compare the same example with inline syntactic annotations:

def x(a: (int,float,complex), b: str, c, d=3:int) -> float:
    return float(d)

Read that second example and ask yourself "what are the parameter names"... hmm, that took a bit to parse, now what happens when those type expressions grow long and convoluted? You get the nice locality of reference, but now you have type-itis all through your parameter lists.

Bonus points that you can do the decorator-based annotations in Python 2.7 and they should work fine in 3.x as well, while the inline static stuff is 3.x only AFAIK. The inline static stuff, however, does allow you to annotate nested parameters and wildcards (named/positional args). I don't know how often those are really going to get type annotations, but I wouldn't consider it a blocker until I saw people really stumbling on it... and then I'd likely tell people to write a couple of more lines of code to solve it...

Just for reference, PyOpenGL has a decorator that does type-annotations in order to get functions pulled out of DLLs via ctypes, that looks like this:

@_f
@_p.types(None,_cs.GLsync,_cs.GLenum,_cs.GLsizei,arrays.GLsizeiArray,arrays.GLintArray)
def glGetSynciv(sync,pname,bufSize,length,values):pass

Where @_f is the "turn it into a ctypes function" and @_p.types( returntype, argtypes ) is the type-annotation code that just adds the type-annotations to the function's dictionary. That works across Python versions, allows IDEs to provide argument names, is fairly concise, etc.

23 Jan 2015 4:09am GMT

Mike C. Fletcher: Type Declarations for Python 2.7

So there's a big flamey thread over on python-list about the Python 3.x type-annotation. Thing is, I can't say the annotations seem all *that* important to the behaviour (static type checking): Consider this Python 2.7 decorator-set:

def _func_annotation_for(func):
    """Retrieve the function annotation for a given function or create it"""
    current = getattr(func,'func_annotation',None)
    if current is None:
        current = func.func_annotation = {}
    return current 
def types( *types, **named ):
    def annotate(function):
        base = _func_annotation_for(function)
        base.update(named)
        if hasattr( function, 'func_code' ): # python 2.x
            names = function.func_code.co_varnames
        else:
            names = function.__code__.co_varnames
        
        for name, typ in zip(names, types):
            base[name] = typ
        return function 
    return annotate
def returns( typ ):
    def annotate(function):
        _func_annotation_for(function)['return'] = typ 
        return function 
    return annotate

That's a few lines of code and would seem to do the bulk of what type annotations do, namely populate a func_annotation dictionary and provide a textual clue as to where the type annotations are for AST or similar tools. It's true that it doesn't look as integrated as the syntax-based annotations, but to my eyes it actually looks cleaner when reading:

@types((int, float, complex), d=int, b=str)
@returns(float)
def x(a, b, c, d=3):
    return float(d)

The extra fluff (type annotations) is separated out and doesn't cloud the parameter list, you can use long type expressions to specify unions-of-types and the like without making it impossible to see your parameters. You can just annotate the functions/parameters you care about... etc. Compare the same example with inline syntactic annotations:

def x(a: (int,float,complex), b: str, c, d=3:int) -> float:
    return float(d)

Read that second example and ask yourself "what are the parameter names"... hmm, that took a bit to parse, now what happens when those type expressions grow long and convoluted? You get the nice locality of reference, but now you have type-itis all through your parameter lists.

Bonus points that you can do the decorator-based annotations in Python 2.7 and they should work fine in 3.x as well, while the inline static stuff is 3.x only AFAIK. The inline static stuff, however, does allow you to annotate nested parameters and wildcards (named/positional args). I don't know how often those are really going to get type annotations, but I wouldn't consider it a blocker until I saw people really stumbling on it... and then I'd likely tell people to write a couple of more lines of code to solve it...

Just for reference, PyOpenGL has a decorator that does type-annotations in order to get functions pulled out of DLLs via ctypes, that looks like this:

@_f
@_p.types(None,_cs.GLsync,_cs.GLenum,_cs.GLsizei,arrays.GLsizeiArray,arrays.GLintArray)
def glGetSynciv(sync,pname,bufSize,length,values):pass

Where @_f is the "turn it into a ctypes function" and @_p.types( returntype, argtypes ) is the type-annotation code that just adds the type-annotations to the function's dictionary. That works across Python versions, allows IDEs to provide argument names, is fairly concise, etc.

23 Jan 2015 4:09am GMT

22 Jan 2015

feedPlanet Python

Caktus Consulting Group: Caktus is looking for a Web Design Director

Over the last two years Caktus' design portfolio has rapidly been growing. We've taken on new projects primarily focused on design and have received community recognition for those efforts. We are happy to have grown our design capabilities to match the level of quality we demand from our Django developers. We have found it's important to have strength on both sides of the table as each side challenges the other and forces the final product of our process to be as high quality as possible.

In an effort to continue to push ourselves and expand our web design skill sets, Caktus is looking to hire a new Web Design Director. We're searching for someone who can do a bit of wireframing and user experience and then has the tools necessary to design and code pages. We're looking for someone who is attune to both form and function and knows where to focus depending on clients' needs. Caktus is committed to doing good in our development communities as well as through the projects that we choose to work on, so we are also interested in finding someone who is engaged in the design community.

If you or someone you know would be a good fit, please apply to the position! If you have any questions get in touch.

22 Jan 2015 1:00pm GMT

Caktus Consulting Group: Caktus is looking for a Web Design Director

Over the last two years Caktus' design portfolio has rapidly been growing. We've taken on new projects primarily focused on design and have received community recognition for those efforts. We are happy to have grown our design capabilities to match the level of quality we demand from our Django developers. We have found it's important to have strength on both sides of the table as each side challenges the other and forces the final product of our process to be as high quality as possible.

In an effort to continue to push ourselves and expand our web design skill sets, Caktus is looking to hire a new Web Design Director. We're searching for someone who can do a bit of wireframing and user experience and then has the tools necessary to design and code pages. We're looking for someone who is attune to both form and function and knows where to focus depending on clients' needs. Caktus is committed to doing good in our development communities as well as through the projects that we choose to work on, so we are also interested in finding someone who is engaged in the design community.

If you or someone you know would be a good fit, please apply to the position! If you have any questions get in touch.

22 Jan 2015 1:00pm GMT

PyPy Development: Faster, more memory efficient and more ordered dictionaries on PyPy

Hello everyone!

As of today, we merged the latest branch that brings better dictionaries to PyPy by default. The work is based on an idea by Raymond Hettinger on python-dev, with prior work done notably in Java. It was done by Maciej Fijałkowski and Armin Rigo, with Laurence Tratt recently prodding us to finish it. (Earlier work going in a similar direction include Alex Gaynor's work on ordered dicts in Topaz, which was also used in the Hippy VM. Each of these pieces of work is itself based on the original dict implementation in RPython, whose origins fade in the Subversion prehistory of PyPy.) Coincidentally, a very similar idea has been implemented in Zend PHP very recently. Zend implementation description.

This post covers the basics of design and implementation as well as some basic benchmarks.

Dictionaries are now ordered!

One surprising part is that the new design, besides being more memory efficient, is ordered by design: it preserves the insertion order. This is not forbidden by the Python language, which allows any order. It makes the collections.OrderedDict subclass much faster than before: it is now a thin subclass of dict. Obviously, we recommend that any portable Python program continues to use OrderedDict when ordering is important. Note that a non-portable program might rely on more: for example, a **keywords argument now receives the keywords in the same order as the one in which they were given in the call. (Whether such a thing might be called a language design change or not is a bit borderline.) The point is that Python programs that work on CPython or previous versions of PyPy should continue to work on PyPy.

There is one exception, though. The iterators of the OrderedDict subclass are now working just like the ones of the dict builtin: they will raise RuntimeError when iterating if the dictionary was modified. In the CPython design, the class OrderedDict explicitly doesn't worry about that, and instead you get some result that might range from correct to incorrect to crashes (i.e. random Python exceptions).

Original PyPy dictionary design

Originally, PyPy dictionaries, as well as CPython dictionaries are implemented as follows (simplified view):

struct dict {
   long num_items;
   dict_entry* items;   /* pointer to array */
}

struct dict_entry {
   long hash;
   PyObject* key;
   PyObject* value;
}

Where items is a sparse array, with 1/3 to 1/2 of the items being NULL. The average space occupied by a dictionary is 3 * WORD * 12/7 plus some small constant (the smallest dict has 8 entries, which is 8 * 3 * WORD + 2 * WORD = 26 WORDs).

New PyPy dictionary design

The new PyPy dictionary is split in two arrays:

struct dict {
    long num_items;
    variable_int *sparse_array;
    dict_entry* compact_array;
}

struct dict_entry {
    long hash;
    PyObject *key;
    PyObject *value;
}

Here, compact_array stores all the items in order of insertion, while sparse_array is a 1/2 to 2/3 full array of integers. The integers themselves are of the smallest size necessary for indexing the compact_array. So if compact_array has less than 256 items, then sparse_array will be made of bytes; if less than 2^16, it'll be two-byte integers; and so on.

This design saves quite a bit of memory. For example, on 64bit systems we can, but almost never, use indexing of more than 4 billion elements; and for small dicts, the extra sparse_array takes very little space. For example a 100 element dict, would be on average for the original design on 64bit: 100 * 12/7 * WORD * 3 =~ 4100 bytes, while on new design it's 100 * 12/7 + 3 * WORD * 100 =~ 2600 bytes, quite a significant saving.

GC friendliness

The obvious benefit of having more compact dictionaries is an increased cache friendliness. In modern CPUs cache misses are much more costly than doing additional simple work, like having an additional level of (in-cache) indirection. Additionally, there is a GC benefit coming from it. When doing a minor collection, the GC has to visit all the GC fields in old objects that can point to young objects. In the case of large arrays, this can prove problematic since the array grows and with each minor collection we need to visit more and more GC pointers. In order to avoid it, large arrays in PyPy employ a technique called "card marking" where the GC only visits "cards" or subsets of arrays that were modified between collections. The problem with dictionaries was that by design modifications in a dictionary occur randomly, hence a lot of cards used to get invalidated. In the new design, however, new items are typically appended to the compact_array, hence invalidate much fewer cards --- which improves GC performance. (The new sparse_array is an array of integers, so it does not suffer from the same problems.)

Deletion

Deleting entries from dictionaries is not very common, but important in a few use cases. To preserve order, when we delete an entry, we mark the entry as removed but don't otherwise shuffle the remaining entries. If we repeat this operation often enough, there will be a lot of removed entries in the (originally compact) array. At this point, we need to do a "packing" operation, which moves all live entries to the start of the array (and then reindexes the sparse array, as the positions changed). This works well, but there are use cases where previously no reindexing was ever needed, so it makes these cases a bit slower (for example when repeatedly adding and removing keys in equal number).

Benchmarks

The PyPy speed benchmarks show mostly small effect, see changes. The microbenchmarks that we did show large improvements on large and very large dictionaries (particularly, building dictionaries of at least a couple 100s of items is now twice faster) and break-even on small ones (between 20% slower and 20% faster depending very much on the usage patterns and sizes of dictionaries). The new dictionaries enable various optimization possibilities which we're going to explore in the near future.

Cheers,
fijal, arigo and the PyPy team


22 Jan 2015 12:31pm GMT

PyPy Development: Faster, more memory efficient and more ordered dictionaries on PyPy

Hello everyone!

As of today, we merged the latest branch that brings better dictionaries to PyPy by default. The work is based on an idea by Raymond Hettinger on python-dev, with prior work done notably in Java. It was done by Maciej Fijałkowski and Armin Rigo, with Laurence Tratt recently prodding us to finish it. (Earlier work going in a similar direction include Alex Gaynor's work on ordered dicts in Topaz, which was also used in the Hippy VM. Each of these pieces of work is itself based on the original dict implementation in RPython, whose origins fade in the Subversion prehistory of PyPy.) Coincidentally, a very similar idea has been implemented in Zend PHP very recently. Zend implementation description.

This post covers the basics of design and implementation as well as some basic benchmarks.

Dictionaries are now ordered!

One surprising part is that the new design, besides being more memory efficient, is ordered by design: it preserves the insertion order. This is not forbidden by the Python language, which allows any order. It makes the collections.OrderedDict subclass much faster than before: it is now a thin subclass of dict. Obviously, we recommend that any portable Python program continues to use OrderedDict when ordering is important. Note that a non-portable program might rely on more: for example, a **keywords argument now receives the keywords in the same order as the one in which they were given in the call. (Whether such a thing might be called a language design change or not is a bit borderline.) The point is that Python programs that work on CPython or previous versions of PyPy should continue to work on PyPy.

There is one exception, though. The iterators of the OrderedDict subclass are now working just like the ones of the dict builtin: they will raise RuntimeError when iterating if the dictionary was modified. In the CPython design, the class OrderedDict explicitly doesn't worry about that, and instead you get some result that might range from correct to incorrect to crashes (i.e. random Python exceptions).

Original PyPy dictionary design

Originally, PyPy dictionaries, as well as CPython dictionaries are implemented as follows (simplified view):

struct dict {
   long num_items;
   dict_entry* items;   /* pointer to array */
}

struct dict_entry {
   long hash;
   PyObject* key;
   PyObject* value;
}

Where items is a sparse array, with 1/3 to 1/2 of the items being NULL. The average space occupied by a dictionary is 3 * WORD * 12/7 plus some small constant (the smallest dict has 8 entries, which is 8 * 3 * WORD + 2 * WORD = 26 WORDs).

New PyPy dictionary design

The new PyPy dictionary is split in two arrays:

struct dict {
    long num_items;
    variable_int *sparse_array;
    dict_entry* compact_array;
}

struct dict_entry {
    long hash;
    PyObject *key;
    PyObject *value;
}

Here, compact_array stores all the items in order of insertion, while sparse_array is a 1/2 to 2/3 full array of integers. The integers themselves are of the smallest size necessary for indexing the compact_array. So if compact_array has less than 256 items, then sparse_array will be made of bytes; if less than 2^16, it'll be two-byte integers; and so on.

This design saves quite a bit of memory. For example, on 64bit systems we can, but almost never, use indexing of more than 4 billion elements; and for small dicts, the extra sparse_array takes very little space. For example a 100 element dict, would be on average for the original design on 64bit: 100 * 12/7 * WORD * 3 =~ 4100 bytes, while on new design it's 100 * 12/7 + 3 * WORD * 100 =~ 2600 bytes, quite a significant saving.

GC friendliness

The obvious benefit of having more compact dictionaries is an increased cache friendliness. In modern CPUs cache misses are much more costly than doing additional simple work, like having an additional level of (in-cache) indirection. Additionally, there is a GC benefit coming from it. When doing a minor collection, the GC has to visit all the GC fields in old objects that can point to young objects. In the case of large arrays, this can prove problematic since the array grows and with each minor collection we need to visit more and more GC pointers. In order to avoid it, large arrays in PyPy employ a technique called "card marking" where the GC only visits "cards" or subsets of arrays that were modified between collections. The problem with dictionaries was that by design modifications in a dictionary occur randomly, hence a lot of cards used to get invalidated. In the new design, however, new items are typically appended to the compact_array, hence invalidate much fewer cards --- which improves GC performance. (The new sparse_array is an array of integers, so it does not suffer from the same problems.)

Deletion

Deleting entries from dictionaries is not very common, but important in a few use cases. To preserve order, when we delete an entry, we mark the entry as removed but don't otherwise shuffle the remaining entries. If we repeat this operation often enough, there will be a lot of removed entries in the (originally compact) array. At this point, we need to do a "packing" operation, which moves all live entries to the start of the array (and then reindexes the sparse array, as the positions changed). This works well, but there are use cases where previously no reindexing was ever needed, so it makes these cases a bit slower (for example when repeatedly adding and removing keys in equal number).

Benchmarks

The PyPy speed benchmarks show mostly small effect, see changes. The microbenchmarks that we did show large improvements on large and very large dictionaries (particularly, building dictionaries of at least a couple 100s of items is now twice faster) and break-even on small ones (between 20% slower and 20% faster depending very much on the usage patterns and sizes of dictionaries). The new dictionaries enable various optimization possibilities which we're going to explore in the near future.

Cheers,
fijal, arigo and the PyPy team


22 Jan 2015 12:31pm GMT

Lennart Regebro: Third goal reached, last hours!

Unexpectedly and amazingly even the third €1200 goal was reached in my funding campaign! The Python community is fantastic! Thanks to everyone!

This is the last day of funding (it says 0 days left), so there is just hours to go (I'm not sure at exactly what time it closes). It's unlikely that the last goal of €2500 will be reached, but I can find ways to put any money over €1200 to good use. For example, I could print copies to give away at PyCon in Montreal. But what to do with that money depends a lot on how much it is.

https://www.fundedbyme.com/en/campaign/5065/supporting-python-3/


Filed under: book, python, python3

22 Jan 2015 8:15am GMT

Lennart Regebro: Third goal reached, last hours!

Unexpectedly and amazingly even the third €1200 goal was reached in my funding campaign! The Python community is fantastic! Thanks to everyone!

This is the last day of funding (it says 0 days left), so there is just hours to go (I'm not sure at exactly what time it closes). It's unlikely that the last goal of €2500 will be reached, but I can find ways to put any money over €1200 to good use. For example, I could print copies to give away at PyCon in Montreal. But what to do with that money depends a lot on how much it is.

https://www.fundedbyme.com/en/campaign/5065/supporting-python-3/


Filed under: book, python, python3

22 Jan 2015 8:15am GMT

Continuum Analytics Blog: Automatic generation of vectorized functions with Blaze and Numba

tl;dr Using Blaze in conjunction with Numba helps speed up code by reducing temporary variables from intermediate computations while allowing you to express computations at a high level.

22 Jan 2015 8:04am GMT

Continuum Analytics Blog: Automatic generation of vectorized functions with Blaze and Numba

tl;dr Using Blaze in conjunction with Numba helps speed up code by reducing temporary variables from intermediate computations while allowing you to express computations at a high level.

22 Jan 2015 8:04am GMT

Nick Coghlan: Abusing Contributors is not OK

As reported in Ars Technica, the ongoing efforts to promote diversity in open source communities came up once more during the plenary Q&A session with Linus Torvalds, Andrew Tridgell, Bdale Garbee and Rusty Russell.

I was there for that session, and found that Linus's response appeared to betray a fundamental misunderstanding of the motives of many of the folks pushing for increased diversity in the open source community, as well as a lack of awareness of the terrible situations that can arise when leaders in a community regularly demonstrate abusive behaviour without suffering any significant consequences (just ask folks like Kathy Sierra, Zoe Quinn, Anita Sarkeesian and Brianna Wu that have been subjected to sustained campaigns of harassment largely for being women that dared to have and express an opinion on the internet).

As the coordinator of the Python Software Foundation's contribution to the linux.conf.au 2015 financial assistance program, and as someone with a deep personal interest in the overall success of the open source community, I feel it is important for me to state explicitly that I consider Linus's level of ignorance around appropriate standards of community conduct to be unacceptable in an open source community leader in 2015.

Linus's defence of his abusive behaviour is that he's "not nice", and "doesn't care about you". He does care deeply about his project, though, and claims to be motivated primarily by wanting that to continue to be successful.

To be completely honest, the momentum behind the Linux juggernaut is now large enough that Linus could likely decide to chuck it all in and spend the rest of his life on a beach sipping cocktails without worrying about open source politics, and people would figure out a way to ensure that Linux continued to thrive and grow without him. Many a successful start-up has made that transition when the founders leave, and there's no good reason to believe an open source community would be fundamentally different in that regard. The transition to a new leadership structure might be a little messy, but the community would almost certainly figure it out.

However, there's still a lot of scope for Linus to influence how fast Linux grows, and on that front his words and actions suggest that he considers being careless in his speech, without regard for the collateral damage his verbal broadsides may be doing to his cause, more important than having the most positive impact he is capable of having on the future growth of the Linux kernel development project and the open source community at large.

It's not (necessarily) about being nice

It may surprise some folks to learn that I don't consider myself a nice human either. My temper is formidable (I just keep it under control most of the time, a task online communication makes easier by providing the ability to walk away from the computer for a while), and any feelings of compassion I have for others are more a product of years of deliberate practice and hanging around with compassionate people than they are any particularly strong innate knack for empathy.

I'm pretty sure that genuinely nice people do exist, and I assume that one of their key motives for creating open, welcoming, inclusive communities is because it's fundamentally the right thing to do. The main reason I exclude myself from my assumed category of "nice people" is that, while I acknowledge that motivation intellectually, it's not really one that I feel viscerally.

Instead, what I do care about, passionately, is helping the best ideas win (where I include "feasible" as part of my definition of "best"). Not the "best ideas from people willing to tolerate extensive personal abuse". The best ideas anyone is willing to share with me, period. And I won't hear those ideas unless I help create environments where all participants are willing to speak up, not just those that are prepared to accept a blistering verbal barrage from a powerful authority figure as a possible consequence of attempting to participate. Those are upsetting enough when they come from random strangers on the internet, when they come from someone with enormous influence not only over you and your future career, but also your entire industry, they can be devastating.

The second order consequences

So there you have it, my not-nice reason for advocating for more welcoming and inclusive open source communities: because, from an engineering standpoint, I consider "has a high level of tolerance for receiving personal abuse from community leaders" to be an extraordinarily stupid filter to apply to your pool of potential contributors.

Exhibiting abusive behaviour as a leader has additional consequences though, and they can be even more problematic: by regularly demonstrating abusive behaviour yourself, you end up normalising harassment within your community in general, both in public and in private.

I believe Linus when he says he doesn't care about who people are or where they're from, only their contributions. I'm the same way - until I've known them for a while, I tend to care about contributors and potential contributors wholesale (i.e. happy people that enjoy the environment they're participating in tend to spend more time engaged, learn faster, produce superior contributions, and more quickly reach the point of being able to contribute independently), rather than retail (i.e. I care about my friends because they're my friends, regardless of context).

But when you're personally abusive as a leader, you also have to take a high level of responsibility for all the folks that look up to you as a role model, and act out the same behaviours you exhibit in public. When you reach this point, the preconditions for participation in your community now include:

  • Willing to tolerate public personal abuse from project leaders
  • Willing to tolerate public personal abuse from the community at large
  • Willing to tolerate personal abuse in private

With clauses like that as part of the definition, the claim of "meritocracy" starts to feel a little shaky, doesn't it? Meritocracy is a fine ideal to strive for, but claiming to have achieved it when you're imposing irrelevant requirements like this is arrogant nonsense.

We're not done yet, though, as this culture of abuse then combines with elitism based on previously acquired knowledge to make it normal to abuse newcomers for still being in the process of learning. I find it hard to conceive of a more effective approach to keeping people from adopting something you're giving away for free than tolerating a community that publicly abuses people for not magically already knowing how to use technology that they may have never even heard of before.

As a result of this perspective, the only time I'll endeavour to eject anyone from a community where I have significant influence is when they're actively creating an unpleasant environment for other participants, and demonstrate no remorse whatsoever regarding the negative impact their actions are having on the overall collaborative effort. I count myself incredibly fortunate to have only had to do this a few times in my life so far, but it's something I believe in strongly enough for it to have been the basis for once deciding to resign from a position paying a six-figure salary at a company I otherwise loved working for. To that company's credit, the abusive leader was let go not long afterwards, but the whole secretive corporate system is rigged such that these toxic "leaders" can usually quickly find new positions elsewhere and hence new subordinates to make miserable - the fact that I'm not willing to name names here for fear of the professional consequences is just one more example of how the system is largely set up to protect abusive leaders rather than holding them to account for the impact their actions have on others.

Ideas and code are still fair game

One of the spurious fears raised against the apparently radical notion of refusing to tolerate personal abuse in a collaborative environment is that adopting civil communication practices somehow means that bad code must then be accepted into the project.

Eliminating personal abuse doesn't mean eliminating rigorous critique of code and ideas. It just means making sure that you are critiquing the code and the ideas, rather than tearing down the person contributing them. It's the difference between "This code isn't any good, here are the problems with it, I'm confident you can do better on your next attempt" (last part optional but usually beneficial when it comes to growing your contributor community) and "This code is terrible, how dare you befoul my presence with it, begone from my sight, worm!".

The latter response may be a funny joke if done in private between close friends, but when it's done in public, in front of a large number of onlookers who don't know either the sender or the recipient personally, it sets an astoundingly bad example as to what a mutually beneficial collaborative peer relationship should look like.

And if you don't have the self-discipline needed to cope with the changing context of our online interactions in the open source community? Well, perhaps you don't yet have the temperament needed to be an open source leader on an internet that is no longer the sole preserve of those of us that are more interested in computers than we are in people. Most of the technical and business press have yet to figure out that they can actually do a bit of investigative journalism to see how well vendor rhetoric aligns with upstream open source engineering activity (frequency of publication is still a far more important performance metric for most journalists than questioning the spin served up in corporate press releases), so the number of folks peering into the open source development fishbowl is only going to grow over time.

It isn't that hard to learn the necessary self-control, though. It's mostly just a matter of taking the time to read each email or code review comment, look for the parts that are about the contributor rather than the code or the design, and remove them before hitting send. And if that means there's nothing left? Then what you were about to send was pure noise, adding nothing useful to the conversation, and hence best left unsent. Doing anything less than this as a community leader is pure self-indulgence, putting your own unwillingness to consider the consequences of your communications ahead of the long term interests of your project. We're only talking about software here, after all - lives aren't on the line when we're deciding how to respond to a particular contribution, so we can afford to take a few moments to review the messages we're about to send and consider how they're likely to be perceived, both by the recipient, and by everyone else observing the exchange.

With any personal abuse removed, you can be as ruthless about critiquing the code and design as you like. Learning not to take critiques of your work personally is a necessary skill to acquire if your ambition is to become a high profile open source developer - the compromises often necessary in the real world of software design and development mean that you will end up shipping things that can legitimately be described as terrible, and you're going to have to learn to be able to say "Yes, I know it's terrible, for reasons X, Y, and Z, and I decided to publish it anyway. If you don't like it, don't use it.". (I highly recommend giving talks about these areas you know are terrible - they're fun to prepare, fun to give, and it's quite entertaining seeing the horrified reactions when people realise I'm not kidding when I say all software is terrible and computers don't actually work, they just fake it fairly well through an ongoing series of horrible hacks built atop other horrible hacks. I'm not surprised the Internet breaks sometimes - given the decades of accumulated legacy hardware and software we're building on and working around, it's thoroughly astonishing that anything technology related ever works at all)

But no matter how harsh your technical critiques get, never forget that there's at least one other human on the far end of that code review or email thread. Even if you don't personally care about them, do you really think it's a good idea to go through life providing large numbers of people with public evidence of why you are a thoroughly unpleasant person to be forced to deal with? As a project leader, do you really think you're going to attract the best and brightest people, who are often free to spend their time and energy however they like, if you set up a sign saying "You must be willing to tolerate extensive personal abuse in order to participate here"?

What can we do about it?

First, and foremost, for those of us that are paid open source community leaders, we can recognise that understanding and motivating our contributors and potential contributors in order to grow our communities is part of our job. If we don't like that, if we'd prefer to be able to "just focus on the code", to the degree where we're not willing to learn how to moderate our own behaviour in accordance with our level of responsibility, then we need to figure out how to reorganise things such that there's someone with better people management and communication skills in a position to act as a buffer between us and our respective communities.

If we instead decide we need to better educate ourselves, then there are plenty of resources available for us to do so. For folks just beginning to explore questions of systemic bias and defaulting to exclusivity, gender-based bias is a good one to start with, by perusing resources like the Feminism 101 section on the Geek Feminism wiki, or (if we have the opportunity) attending an Ada Initiative Ally Skills workshop.

And if we do acknowledge the importance of this work, then we can use our influence to help it continue, whether that's by sponsoring educational workshops, supporting financial assistance programs, ensuring suitable codes of conduct are in place for our events and online communities, supporting programs like the GNOME Outreach Program for Women, or organisations like the Ada Initiative, and so on, and so forth.

For those of us that aren't community leaders, then one of the most effective things we can do is vote with our feet: at last count, there are over a million open source projects in existence, many of them are run in such a way that participating in them is almost always a sheer pleasure, and if no existing community grabs your interest, you always have the option of starting your own.

Personal enjoyment is only one reason for participating in open source though, and professional obligations or personal needs may bring us into contact with project leaders and contributors that currently consider personal abuse to be an acceptable way of interacting with their peers in a collaborative context. If leaving isn't a viable option, then what can we do?

Firstly, the options I suggest above for community leaders are actually good options for any participants in the open source community that view the overall growth and success of the free and open source software ethos as being more important than any one individual's personal pride or reluctance to educate themselves about issues that don't affect them personally.

Secondly, we can hold our leaders to account. When community leaders give presentations at community events, especially when presenting on community management topics, feel free to ask the following questions (or variations on these themes):

  • Are we as community leaders aware of the impact current and historical structural inequalities have on the demographics of our community?
  • What have we done recently as individuals to improve our understanding of these issues and their consequences?
  • What have we done recently as community leaders to attempt to counter the systemic biases adversely affecting the demographics of our communities?

These are questions that open source community leaders should be able to answer. When we can't, I guess the silver lining is that it means we have plenty of scope to get better at what we do. For members of vulnerable groups, an inability for leaders to answer these questions is also a strong sign as to which communities may not yet be able to provide safe spaces for you to participate without experiencing harassment over your identity rather than being critiqued solely based on the quality of your work.

If you ask these questions, you will get people complaining about bringing politics into a technical environment. The folks complaining are simply wrong, as the single most important factor driving the quality of our technology is the quality of our thinking. Assuming we have attained meritocracy (aka "the most effective collaborative environment possible") is sheer foolishness, when a wide array of systemic biases remain in place that work to reduce the depth, breadth, and hence quality, of our collective thinking.

Update 22 Jan, 2015: Minor typo and grammar fixes

22 Jan 2015 5:57am GMT

Nick Coghlan: Abusing Contributors is not OK

As reported in Ars Technica, the ongoing efforts to promote diversity in open source communities came up once more during the plenary Q&A session with Linus Torvalds, Andrew Tridgell, Bdale Garbee and Rusty Russell.

I was there for that session, and found that Linus's response appeared to betray a fundamental misunderstanding of the motives of many of the folks pushing for increased diversity in the open source community, as well as a lack of awareness of the terrible situations that can arise when leaders in a community regularly demonstrate abusive behaviour without suffering any significant consequences (just ask folks like Kathy Sierra, Zoe Quinn, Anita Sarkeesian and Brianna Wu that have been subjected to sustained campaigns of harassment largely for being women that dared to have and express an opinion on the internet).

As the coordinator of the Python Software Foundation's contribution to the linux.conf.au 2015 financial assistance program, and as someone with a deep personal interest in the overall success of the open source community, I feel it is important for me to state explicitly that I consider Linus's level of ignorance around appropriate standards of community conduct to be unacceptable in an open source community leader in 2015.

Linus's defence of his abusive behaviour is that he's "not nice", and "doesn't care about you". He does care deeply about his project, though, and claims to be motivated primarily by wanting that to continue to be successful.

To be completely honest, the momentum behind the Linux juggernaut is now large enough that Linus could likely decide to chuck it all in and spend the rest of his life on a beach sipping cocktails without worrying about open source politics, and people would figure out a way to ensure that Linux continued to thrive and grow without him. Many a successful start-up has made that transition when the founders leave, and there's no good reason to believe an open source community would be fundamentally different in that regard. The transition to a new leadership structure might be a little messy, but the community would almost certainly figure it out.

However, there's still a lot of scope for Linus to influence how fast Linux grows, and on that front his words and actions suggest that he considers being careless in his speech, without regard for the collateral damage his verbal broadsides may be doing to his cause, more important than having the most positive impact he is capable of having on the future growth of the Linux kernel development project and the open source community at large.

It's not (necessarily) about being nice

It may surprise some folks to learn that I don't consider myself a nice human either. My temper is formidable (I just keep it under control most of the time, a task online communication makes easier by providing the ability to walk away from the computer for a while), and any feelings of compassion I have for others are more a product of years of deliberate practice and hanging around with compassionate people than they are any particularly strong innate knack for empathy.

I'm pretty sure that genuinely nice people do exist, and I assume that one of their key motives for creating open, welcoming, inclusive communities is because it's fundamentally the right thing to do. The main reason I exclude myself from my assumed category of "nice people" is that, while I acknowledge that motivation intellectually, it's not really one that I feel viscerally.

Instead, what I do care about, passionately, is helping the best ideas win (where I include "feasible" as part of my definition of "best"). Not the "best ideas from people willing to tolerate extensive personal abuse". The best ideas anyone is willing to share with me, period. And I won't hear those ideas unless I help create environments where all participants are willing to speak up, not just those that are prepared to accept a blistering verbal barrage from a powerful authority figure as a possible consequence of attempting to participate. Those are upsetting enough when they come from random strangers on the internet, when they come from someone with enormous influence not only over you and your future career, but also your entire industry, they can be devastating.

The second order consequences

So there you have it, my not-nice reason for advocating for more welcoming and inclusive open source communities: because, from an engineering standpoint, I consider "has a high level of tolerance for receiving personal abuse from community leaders" to be an extraordinarily stupid filter to apply to your pool of potential contributors.

Exhibiting abusive behaviour as a leader has additional consequences though, and they can be even more problematic: by regularly demonstrating abusive behaviour yourself, you end up normalising harassment within your community in general, both in public and in private.

I believe Linus when he says he doesn't care about who people are or where they're from, only their contributions. I'm the same way - until I've known them for a while, I tend to care about contributors and potential contributors wholesale (i.e. happy people that enjoy the environment they're participating in tend to spend more time engaged, learn faster, produce superior contributions, and more quickly reach the point of being able to contribute independently), rather than retail (i.e. I care about my friends because they're my friends, regardless of context).

But when you're personally abusive as a leader, you also have to take a high level of responsibility for all the folks that look up to you as a role model, and act out the same behaviours you exhibit in public. When you reach this point, the preconditions for participation in your community now include:

  • Willing to tolerate public personal abuse from project leaders
  • Willing to tolerate public personal abuse from the community at large
  • Willing to tolerate personal abuse in private

With clauses like that as part of the definition, the claim of "meritocracy" starts to feel a little shaky, doesn't it? Meritocracy is a fine ideal to strive for, but claiming to have achieved it when you're imposing irrelevant requirements like this is arrogant nonsense.

We're not done yet, though, as this culture of abuse then combines with elitism based on previously acquired knowledge to make it normal to abuse newcomers for still being in the process of learning. I find it hard to conceive of a more effective approach to keeping people from adopting something you're giving away for free than tolerating a community that publicly abuses people for not magically already knowing how to use technology that they may have never even heard of before.

As a result of this perspective, the only time I'll endeavour to eject anyone from a community where I have significant influence is when they're actively creating an unpleasant environment for other participants, and demonstrate no remorse whatsoever regarding the negative impact their actions are having on the overall collaborative effort. I count myself incredibly fortunate to have only had to do this a few times in my life so far, but it's something I believe in strongly enough for it to have been the basis for once deciding to resign from a position paying a six-figure salary at a company I otherwise loved working for. To that company's credit, the abusive leader was let go not long afterwards, but the whole secretive corporate system is rigged such that these toxic "leaders" can usually quickly find new positions elsewhere and hence new subordinates to make miserable - the fact that I'm not willing to name names here for fear of the professional consequences is just one more example of how the system is largely set up to protect abusive leaders rather than holding them to account for the impact their actions have on others.

Ideas and code are still fair game

One of the spurious fears raised against the apparently radical notion of refusing to tolerate personal abuse in a collaborative environment is that adopting civil communication practices somehow means that bad code must then be accepted into the project.

Eliminating personal abuse doesn't mean eliminating rigorous critique of code and ideas. It just means making sure that you are critiquing the code and the ideas, rather than tearing down the person contributing them. It's the difference between "This code isn't any good, here are the problems with it, I'm confident you can do better on your next attempt" (last part optional but usually beneficial when it comes to growing your contributor community) and "This code is terrible, how dare you befoul my presence with it, begone from my sight, worm!".

The latter response may be a funny joke if done in private between close friends, but when it's done in public, in front of a large number of onlookers who don't know either the sender or the recipient personally, it sets an astoundingly bad example as to what a mutually beneficial collaborative peer relationship should look like.

And if you don't have the self-discipline needed to cope with the changing context of our online interactions in the open source community? Well, perhaps you don't yet have the temperament needed to be an open source leader on an internet that is no longer the sole preserve of those of us that are more interested in computers than we are in people. Most of the technical and business press have yet to figure out that they can actually do a bit of investigative journalism to see how well vendor rhetoric aligns with upstream open source engineering activity (frequency of publication is still a far more important performance metric for most journalists than questioning the spin served up in corporate press releases), so the number of folks peering into the open source development fishbowl is only going to grow over time.

It isn't that hard to learn the necessary self-control, though. It's mostly just a matter of taking the time to read each email or code review comment, look for the parts that are about the contributor rather than the code or the design, and remove them before hitting send. And if that means there's nothing left? Then what you were about to send was pure noise, adding nothing useful to the conversation, and hence best left unsent. Doing anything less than this as a community leader is pure self-indulgence, putting your own unwillingness to consider the consequences of your communications ahead of the long term interests of your project. We're only talking about software here, after all - lives aren't on the line when we're deciding how to respond to a particular contribution, so we can afford to take a few moments to review the messages we're about to send and consider how they're likely to be perceived, both by the recipient, and by everyone else observing the exchange.

With any personal abuse removed, you can be as ruthless about critiquing the code and design as you like. Learning not to take critiques of your work personally is a necessary skill to acquire if your ambition is to become a high profile open source developer - the compromises often necessary in the real world of software design and development mean that you will end up shipping things that can legitimately be described as terrible, and you're going to have to learn to be able to say "Yes, I know it's terrible, for reasons X, Y, and Z, and I decided to publish it anyway. If you don't like it, don't use it.". (I highly recommend giving talks about these areas you know are terrible - they're fun to prepare, fun to give, and it's quite entertaining seeing the horrified reactions when people realise I'm not kidding when I say all software is terrible and computers don't actually work, they just fake it fairly well through an ongoing series of horrible hacks built atop other horrible hacks. I'm not surprised the Internet breaks sometimes - given the decades of accumulated legacy hardware and software we're building on and working around, it's thoroughly astonishing that anything technology related ever works at all)

But no matter how harsh your technical critiques get, never forget that there's at least one other human on the far end of that code review or email thread. Even if you don't personally care about them, do you really think it's a good idea to go through life providing large numbers of people with public evidence of why you are a thoroughly unpleasant person to be forced to deal with? As a project leader, do you really think you're going to attract the best and brightest people, who are often free to spend their time and energy however they like, if you set up a sign saying "You must be willing to tolerate extensive personal abuse in order to participate here"?

What can we do about it?

First, and foremost, for those of us that are paid open source community leaders, we can recognise that understanding and motivating our contributors and potential contributors in order to grow our communities is part of our job. If we don't like that, if we'd prefer to be able to "just focus on the code", to the degree where we're not willing to learn how to moderate our own behaviour in accordance with our level of responsibility, then we need to figure out how to reorganise things such that there's someone with better people management and communication skills in a position to act as a buffer between us and our respective communities.

If we instead decide we need to better educate ourselves, then there are plenty of resources available for us to do so. For folks just beginning to explore questions of systemic bias and defaulting to exclusivity, gender-based bias is a good one to start with, by perusing resources like the Feminism 101 section on the Geek Feminism wiki, or (if we have the opportunity) attending an Ada Initiative Ally Skills workshop.

And if we do acknowledge the importance of this work, then we can use our influence to help it continue, whether that's by sponsoring educational workshops, supporting financial assistance programs, ensuring suitable codes of conduct are in place for our events and online communities, supporting programs like the GNOME Outreach Program for Women, or organisations like the Ada Initiative, and so on, and so forth.

For those of us that aren't community leaders, then one of the most effective things we can do is vote with our feet: at last count, there are over a million open source projects in existence, many of them are run in such a way that participating in them is almost always a sheer pleasure, and if no existing community grabs your interest, you always have the option of starting your own.

Personal enjoyment is only one reason for participating in open source though, and professional obligations or personal needs may bring us into contact with project leaders and contributors that currently consider personal abuse to be an acceptable way of interacting with their peers in a collaborative context. If leaving isn't a viable option, then what can we do?

Firstly, the options I suggest above for community leaders are actually good options for any participants in the open source community that view the overall growth and success of the free and open source software ethos as being more important than any one individual's personal pride or reluctance to educate themselves about issues that don't affect them personally.

Secondly, we can hold our leaders to account. When community leaders give presentations at community events, especially when presenting on community management topics, feel free to ask the following questions (or variations on these themes):

  • Are we as community leaders aware of the impact current and historical structural inequalities have on the demographics of our community?
  • What have we done recently as individuals to improve our understanding of these issues and their consequences?
  • What have we done recently as community leaders to attempt to counter the systemic biases adversely affecting the demographics of our communities?

These are questions that open source community leaders should be able to answer. When we can't, I guess the silver lining is that it means we have plenty of scope to get better at what we do. For members of vulnerable groups, an inability for leaders to answer these questions is also a strong sign as to which communities may not yet be able to provide safe spaces for you to participate without experiencing harassment over your identity rather than being critiqued solely based on the quality of your work.

If you ask these questions, you will get people complaining about bringing politics into a technical environment. The folks complaining are simply wrong, as the single most important factor driving the quality of our technology is the quality of our thinking. Assuming we have attained meritocracy (aka "the most effective collaborative environment possible") is sheer foolishness, when a wide array of systemic biases remain in place that work to reduce the depth, breadth, and hence quality, of our collective thinking.

Update 22 Jan, 2015: Minor typo and grammar fixes

22 Jan 2015 5:57am GMT

Wingware News: Wing IDE 5.1: January 22, 2015

Wing IDE 5.1 adds multi-process and child process debugging, syntax highlighting in the shells, persistent time-stamped unit test results, auto-conversion of indents on paste, an XCode keyboard personality, support for Flask and Django 1.7, and many other minor features and improvements.

22 Jan 2015 1:00am GMT

Wingware News: Wing IDE 5.1: January 22, 2015

Wing IDE 5.1 adds multi-process and child process debugging, syntax highlighting in the shells, persistent time-stamped unit test results, auto-conversion of indents on paste, an XCode keyboard personality, support for Flask and Django 1.7, and many other minor features and improvements.

22 Jan 2015 1:00am GMT

21 Jan 2015

feedPlanet Python

Omaha Python Users Group: Meeting Tonight! 2015-01-21

Location - Gordmans Office Building in Aksarben thanks to Aaron Keck.

Meeting starts at 6pm, Wednesday, 1/21/15

Parking and entry details:

The building is the northwest corner of 67th and Frances and the Gordmans entrance is next to the "g" sign, about midway along the East side of the building. There's parking directly out front, but it sometimes fills up in the evenings. The garage around back is open to the public after 5:30 or 6 as well.

The building doors lock at 5, so Aaron will be standing by to badge people in starting around 5:45. If you're running late, or early, just shoot him or the list an email.

21 Jan 2015 9:57pm GMT

Omaha Python Users Group: Meeting Tonight! 2015-01-21

Location - Gordmans Office Building in Aksarben thanks to Aaron Keck.

Meeting starts at 6pm, Wednesday, 1/21/15

Parking and entry details:

The building is the northwest corner of 67th and Frances and the Gordmans entrance is next to the "g" sign, about midway along the East side of the building. There's parking directly out front, but it sometimes fills up in the evenings. The garage around back is open to the public after 5:30 or 6 as well.

The building doors lock at 5, so Aaron will be standing by to badge people in starting around 5:45. If you're running late, or early, just shoot him or the list an email.

21 Jan 2015 9:57pm GMT

10 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: King Willams Town Bahnhof

Gestern musste ich morgens zur Station nach KWT um unsere Rerservierten Bustickets für die Weihnachtsferien in Capetown abzuholen. Der Bahnhof selber ist seit Dezember aus kostengründen ohne Zugverbindung - aber Translux und co - die langdistanzbusse haben dort ihre Büros.


Größere Kartenansicht




© benste CC NC SA

10 Nov 2011 10:57am GMT

09 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein

Niemand ist besorgt um so was - mit dem Auto fährt man einfach durch, und in der City - nahe Gnobie- "ne das ist erst gefährlich wenn die Feuerwehr da ist" - 30min später auf dem Rückweg war die Feuerwehr da.




© benste CC NC SA

09 Nov 2011 8:25pm GMT

08 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Brai Party

Brai = Grillabend o.ä.

Die möchte gern Techniker beim Flicken ihrer SpeakOn / Klinke Stecker Verzweigungen...

Die Damen "Mamas" der Siedlung bei der offiziellen Eröffnungsrede

Auch wenn weniger Leute da waren als erwartet, Laute Musik und viele Leute ...

Und natürlich ein Feuer mit echtem Holz zum Grillen.

© benste CC NC SA

08 Nov 2011 2:30pm GMT

07 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Lumanyano Primary

One of our missions was bringing Katja's Linux Server back to her room. While doing that we saw her new decoration.

Björn, Simphiwe carried the PC to Katja's school


© benste CC NC SA

07 Nov 2011 2:00pm GMT

06 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nelisa Haircut

Today I went with Björn to Needs Camp to Visit Katja's guest family for a special Party. First of all we visited some friends of Nelisa - yeah the one I'm working with in Quigney - Katja's guest fathers sister - who did her a haircut.

African Women usually get their hair done by arranging extensions and not like Europeans just cutting some hair.

In between she looked like this...

And then she was done - looks amazing considering the amount of hair she had last week - doesn't it ?

© benste CC NC SA

06 Nov 2011 7:45pm GMT

05 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Mein Samstag

Irgendwie viel mir heute auf das ich meine Blogposts mal ein bischen umstrukturieren muss - wenn ich immer nur von neuen Plätzen berichte, dann müsste ich ja eine Rundreise machen. Hier also mal ein paar Sachen aus meinem heutigen Alltag.

Erst einmal vorweg, Samstag zählt zumindest für uns Voluntäre zu den freien Tagen.

Dieses Wochenende sind nur Rommel und ich auf der Farm - Katja und Björn sind ja mittlerweile in ihren Einsatzstellen, und meine Mitbewohner Kyle und Jonathan sind zu Hause in Grahamstown - sowie auch Sipho der in Dimbaza wohnt.
Robin, die Frau von Rommel ist in Woodie Cape - schon seit Donnerstag um da ein paar Sachen zur erledigen.
Naja wie dem auch sei heute morgen haben wir uns erstmal ein gemeinsames Weetbix/Müsli Frühstück gegönnt und haben uns dann auf den Weg nach East London gemacht. 2 Sachen waren auf der Checkliste Vodacom, Ethienne (Imobilienmakler) außerdem auf dem Rückweg die fehlenden Dinge nach NeedsCamp bringen.

Nachdem wir gerade auf der Dirtroad losgefahren sind mussten wir feststellen das wir die Sachen für Needscamp und Ethienne nicht eingepackt hatten aber die Pumpe für die Wasserversorgung im Auto hatten.

Also sind wir in EastLondon ersteinmal nach Farmerama - nein nicht das onlinespiel farmville - sondern einen Laden mit ganz vielen Sachen für eine Farm - in Berea einem nördlichen Stadteil gefahren.

In Farmerama haben wir uns dann beraten lassen für einen Schnellverschluss der uns das leben mit der Pumpe leichter machen soll und außerdem eine leichtere Pumpe zur Reperatur gebracht, damit es nicht immer so ein großer Aufwand ist, wenn mal wieder das Wasser ausgegangen ist.

Fego Caffé ist in der Hemmingways Mall, dort mussten wir und PIN und PUK einer unserer Datensimcards geben lassen, da bei der PIN Abfrage leider ein zahlendreher unterlaufen ist. Naja auf jeden Fall speichern die Shops in Südafrika so sensible Daten wie eine PUK - die im Prinzip zugang zu einem gesperrten Phone verschafft.

Im Cafe hat Rommel dann ein paar online Transaktionen mit dem 3G Modem durchgeführt, welches ja jetzt wieder funktionierte - und übrigens mittlerweile in Ubuntu meinem Linuxsystem perfekt klappt.

Nebenbei bin ich nach 8ta gegangen um dort etwas über deren neue Deals zu erfahren, da wir in einigen von Hilltops Centern Internet anbieten wollen. Das Bild zeigt die Abdeckung UMTS in NeedsCamp Katjas Ort. 8ta ist ein neuer Telefonanbieter von Telkom, nachdem Vodafone sich Telkoms anteile an Vodacom gekauft hat müssen die komplett neu aufbauen.
Wir haben uns dazu entschieden mal eine kostenlose Prepaidkarte zu testen zu organisieren, denn wer weis wie genau die Karte oben ist ... Bevor man einen noch so billigen Deal für 24 Monate signed sollte man wissen obs geht.

Danach gings nach Checkers in Vincent, gesucht wurden zwei Hotplates für WoodyCape - R 129.00 eine - also ca. 12€ für eine zweigeteilte Kochplatte.
Wie man sieht im Hintergrund gibts schon Weihnachtsdeko - Anfang November und das in Südafrika bei sonnig warmen min- 25°C

Mittagessen haben wir uns bei einem Pakistanischen Curry Imbiss gegönnt - sehr empfehlenswert !
Naja und nachdem wir dann vor ner Stunde oder so zurück gekommen sind habe ich noch den Kühlschrank geputzt den ich heute morgen zum defrosten einfach nach draußen gestellt hatte. Jetzt ist der auch mal wieder sauber und ohne 3m dicke Eisschicht...

Morgen ... ja darüber werde ich gesondert berichten ... aber vermutlich erst am Montag, denn dann bin ich nochmal wieder in Quigney(East London) und habe kostenloses Internet.

© benste CC NC SA

05 Nov 2011 4:33pm GMT

31 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Sterkspruit Computer Center

Sterkspruit is one of Hilltops Computer Centres in the far north of Eastern Cape. On the trip to J'burg we've used the opportunity to take a look at the centre.

Pupils in the big classroom


The Trainer


School in Countryside


Adult Class in the Afternoon


"Town"


© benste CC NC SA

31 Oct 2011 4:58pm GMT

Benedict Stein: Technical Issues

What are you doing in an internet cafe if your ADSL and Faxline has been discontinued before months end. Well my idea was sitting outside and eating some ice cream.
At least it's sunny and not as rainy as on the weekend.


© benste CC NC SA

31 Oct 2011 3:11pm GMT

30 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nellis Restaurant

For those who are traveling through Zastron - there is a very nice Restaurant which is serving delicious food at reasanable prices.
In addition they're selling home made juices jams and honey.




interior


home made specialities - the shop in the shop


the Bar


© benste CC NC SA

30 Oct 2011 4:47pm GMT

29 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: The way back from J'burg

Having the 10 - 12h trip from J'burg back to ELS I was able to take a lot of pcitures including these different roadsides

Plain Street


Orange River in its beginngings (near Lesotho)


Zastron Anglican Church


The Bridge in Between "Free State" and Eastern Cape next to Zastron


my new Background ;)


If you listen to GoogleMaps you'll end up traveling 50km of gravel road - as it was just renewed we didn't have that many problems and saved 1h compared to going the official way with all it's constructions sites




Freeway


getting dark


© benste CC NC SA

29 Oct 2011 4:23pm GMT

28 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Wie funktioniert eigentlich eine Baustelle ?

Klar einiges mag anders sein, vieles aber gleich - aber ein in Deutschland täglich übliches Bild einer Straßenbaustelle - wie läuft das eigentlich in Südafrika ?

Ersteinmal vorweg - NEIN keine Ureinwohner die mit den Händen graben - auch wenn hier mehr Manpower genutzt wird - sind sie fleißig mit Technologie am arbeiten.

Eine ganz normale "Bundesstraße"


und wie sie erweitert wird


gaaaanz viele LKWs


denn hier wird eine Seite über einen langen Abschnitt komplett gesperrt, so das eine Ampelschaltung mit hier 45 Minuten Wartezeit entsteht


Aber wenigstens scheinen die ihren Spaß zu haben ;) - Wie auch wir denn gücklicher Weise mussten wir nie länger als 10 min. warten.

© benste CC NC SA

28 Oct 2011 4:20pm GMT