24 Apr 2014

feedPlanet Python

PyCharm: The PyCharm 3.4 Early Access Program has begun

animation

The JetBrains PyCharm team is pleased to announce the start of the Early Access Program (EAP) for PyCharm 3.4 and is ready to introduce the brand-new build 135.763 for your evaluation. It's already here, available for download from the Early Access Preview page. As any new PyCharm 3.x release (starting from PyCharm 3.0), the PyCharm 3.4 EAP build is available in two editions - free/open source PyCharm Community Edition and full-fledged PyCharm Professional Edition. While both of them are available for download and use at no charge, PyCharm Professional Edition has a 30-day evaluation period, which means PyCharm 3.4 Professional Edition EAP build 135.763 comes with a 30-day time-limited license as well.

The PyCharm 3.4 EAP build is not in the final product release stage and might contain severe bugs, so it's supposed to be installed along with your current PyCharm installation for evaluation and checking out new features. No patch update for this EAP build will be available from within the IDE.

We'd like to encourage you to try this brand-new preview build and give us your valuable feedback in our public issue tracker. This will help us make the final release of PyCharm 3.4 more stable and less buggy.

PyCharm 3.4 EAP build 135.763 includes a lot of new features, bug fixes, and improvements in different subsystems. It also delivers a lot of unique PyCharm-specific features. Meanwhile, PyCharm users also have access to a number of recently announced features and benefits from other JetBrains products like WebStorm 8 and IntelliJ IDEA 13.1. For example, advanced built-in AngularJS support and Sublime Text style multiple selection:

out0033

So here are the PyCharm-specific features:

new_proj_interpr

Now you can choose existing, add a new local or remote python interpreter and even create virtualenv right on the project creation stage without any need to sneak into the PyCharm settings.

templates

templates2

PyCharm 3.4 EAP also incorporates all new WebStorm 8 features (read more on the What's new in WebStorm 8 page), so from the WebStorm side we are ready to deliver:

AngularJS-large.png

And finally, the IntelliJ platform brings us many features including:

With Multiple Selection, we can now put a cursor in multiple locations in a file and write code simultaneously in these positions. Press and hold "Alt" on the keyboard and use the mouse to select the locations of the cursors:

animation_web

Download PyCharm 3.4 EAP build 135.763 for your platform from the project EAP page and please report any bugs and feature request to our Issue Tracker.

Remember to install the .zip & .tar.gz versions into a completely empty folder. Do not just unpack over your previous version!

Develop with pleasure!
-PyCharm team

24 Apr 2014 10:50pm GMT

PyCharm: The PyCharm 3.4 Early Access Program has begun

animation

The JetBrains PyCharm team is pleased to announce the start of the Early Access Program (EAP) for PyCharm 3.4 and is ready to introduce the brand-new build 135.763 for your evaluation. It's already here, available for download from the Early Access Preview page. As any new PyCharm 3.x release (starting from PyCharm 3.0), the PyCharm 3.4 EAP build is available in two editions - free/open source PyCharm Community Edition and full-fledged PyCharm Professional Edition. While both of them are available for download and use at no charge, PyCharm Professional Edition has a 30-day evaluation period, which means PyCharm 3.4 Professional Edition EAP build 135.763 comes with a 30-day time-limited license as well.

The PyCharm 3.4 EAP build is not in the final product release stage and might contain severe bugs, so it's supposed to be installed along with your current PyCharm installation for evaluation and checking out new features. No patch update for this EAP build will be available from within the IDE.

We'd like to encourage you to try this brand-new preview build and give us your valuable feedback in our public issue tracker. This will help us make the final release of PyCharm 3.4 more stable and less buggy.

PyCharm 3.4 EAP build 135.763 includes a lot of new features, bug fixes, and improvements in different subsystems. It also delivers a lot of unique PyCharm-specific features. Meanwhile, PyCharm users also have access to a number of recently announced features and benefits from other JetBrains products like WebStorm 8 and IntelliJ IDEA 13.1. For example, advanced built-in AngularJS support and Sublime Text style multiple selection:

out0033

So here are the PyCharm-specific features:

new_proj_interpr

Now you can choose existing, add a new local or remote python interpreter and even create virtualenv right on the project creation stage without any need to sneak into the PyCharm settings.

templates

templates2

PyCharm 3.4 EAP also incorporates all new WebStorm 8 features (read more on the What's new in WebStorm 8 page), so from the WebStorm side we are ready to deliver:

AngularJS-large.png

And finally, the IntelliJ platform brings us many features including:

With Multiple Selection, we can now put a cursor in multiple locations in a file and write code simultaneously in these positions. Press and hold "Alt" on the keyboard and use the mouse to select the locations of the cursors:

animation_web

Download PyCharm 3.4 EAP build 135.763 for your platform from the project EAP page and please report any bugs and feature request to our Issue Tracker.

Remember to install the .zip & .tar.gz versions into a completely empty folder. Do not just unpack over your previous version!

Develop with pleasure!
-PyCharm team

24 Apr 2014 10:50pm GMT

Reinout van Rees: Python/django programmeren in hartje Utrecht (NL)

English note: I'm writing this in Dutch as it is for a job opening at our company. Call us old-fashioned, but we're all talking Dutch at the office :-)

We (= Nelen & Schuurmans) hebben weer versterking nodig van ons team Python programmeurs. Je kans om met Reinout te werken want de zaken gaan goed en er is veel werk te verstouwen :-) Laatst is een van onze ervaren programmeurs voor zichzelf begonnen en daarom gooi ik hier op mijn blog weer een hengeltje uit: we zijn specifiek ook op zoek naar een ervaren Python programmeur.

Ervaren? Nou, gewoon, een stukje kwaliteit en kunde en kennis inbrengen. Het mooiste compliment dat ik ooit van een collega (hoi Roland!) kreeg was dat 'ie mijn code altijd zo lekker leesbaar vond. Duidelijke variabelenamen, netjes gestructureerd, netjes pep8. Zoiets. Gewoon dat als je code schrijft dat het dan z'n werk goed doet en dat het niet afgeraffeld is. Dat je geheel vrijwillig de README.rst en de CHANGES.rst up to date houdt. En dat je na een maandje de eerste collega's aan je bureau krijgt met "kun je me even helpen met...". En dat je op een gegeven moment met ik-weet-niet-wat iets heel handigs hebt geautomatiseerd ofzo. Vanuit jezelf.

Grote bonuspunten als je veel met open source bezig bent geweest. Github, mailinglijsten, stackoverflow, enz. Veel van wat we als bedrijf doen is open source (vaak GPL, dan is het wel lekker open source maar kan tegelijkertijd de eventuele concurrent er niet zonder licentieproblemen een closed source slaatje uit slaan).

Het karakter van het bedrijf? We zijn geen puur IT bedrijf, het is echt een mix van IT en vakinhoud. Qua personele verhouding 1:2. Vakinhoudelijk is het waterveiligheid, waterkwaliteit, waterbeheer en vooral dus water. IT is een echte force multiplier voor ons. Wat we qua IT doen en kunnen versterkt de vakinhoudelijke kant enorm.

En we doen inmiddels best veel gave dingen. 3di is een compleet overstromingssimulatiepakket. Inclusief webinterface om het live aan te sturen. (Iemand heeft ook een video van het ontwikkelteam gemaakt). En kijk bijvoorbeeld naar http://regenradar.lizard.net/ (moet het wel net geregend hebben, zie anders deze video), da's op zich geen concurrent voor buienradar.nl maar geeft wel aan dat we neerslagdata hebben. En ook historische neerslagdata. En we integreren het met de rest van de data. Bij twee waterschappen berekenen we bijvoorbeeld aan de hand van neerslagvoorspellingen, meetgegevens, windrichting en getijde-voorspellingen hoe hard gemalen aangezet moeten worden. En dat gedetailleerde hoogtebestand dat laatst in het nieuws was (AHN2)? https://raster.lizard.net/wms/demo is een demo, we integreren het inmiddels in veel sites en we gebruiken het ook als achtergronddata voor dat 3Di dat ik eerder noemde.

Essentieel voor het karakter van het bedrijf is dat we allemaal hoog opgeleid zijn. Bijna iedereen universitair. En ik ben niet de enige dr. die er rondloopt: 8 van de 50 ofzo. Dat is eigenlijk de kern van het bedrijf: zet allemaal slimme, ijverige lui bij elkaar met een focus op waterveiligheid en er komen mooie (en winstgevende) dingen uit. Je hebt veel vrijheid (vertaald: het is soms een zooitje). Ruimte voor eigen initiatief.

Qua werkomgeving? Hartje Utrecht. Regelmatig halen we een ijsje bij dat ijskraampje op de Oude Gracht: dat is 1 minuut lopen. Goed bereikbaar: 10 minuten lopen vanaf het centraal station. Lunch enzo? Gezamenlijk in de kelder aan tafel. Vers brood van de bakker. Goed geregeld, wat dat betreft. Verder zijn veel dingen relatief goedkoop geregeld. Geen apple beeldschermen op het bureau zullen we maar zeggen. Voordeel: elk jaar hebben we tot nu toe winst gedraaid. Da's in deze tijd ook een geruststellende gedachte.

Werkinhoudelijk? Veel Python. Veel ook aan de web kant: Django. Er moet ook geautomatiseerd worden, dus Fabric, Ansible, buildout, enz. Her en der zit nog een snufje Java of Fortran. Natuurlijk ook veel javascript (recent ook steeds meer angularjs). Nette Python packages, changelog, readme, buildout, grunt/bower enz. Onze code staat op github. Branches, pull requests, code review. Tests kunnen we meer van gebruiken, maar we hebben ook al een aardige stapel. Zelfs javascript wordt deels getest. (Dus... qua tests kunnen we nog een extra opvoeder gebruiken). We hebben in twee datacenters een rack volgepropt, dus we hebben en vette VMWare omgeving met veel virtual machines. Best gaaf geregeld.

Het leukste: we hebben een groot IT team. Ongeveer 10 Python programmeurs (met bakken css/javascript/whatever kennis erbij). Da's binnen Nederland best een grote club. Volgens mij horen we gewoon bij de grootste Python bedrijven binnen Nederland. Je kan bij ons veel leren en sparren en experimenteren en helpen en samenwerken. Gaaf.

Interesse? Meer info nodig? Je kan kijken naar alles wat ik met "nelenschuurmans" getagt heb op dit weblog. Je kan mij mailen op reinout@vanrees.org.

  • De officiele vacature staat hier: http://www.nelen-schuurmans.nl/NL/bedrijfsprofiel/carriere . Het emailadres voor de sollicitatie staat daar ook bij. Doe me een plezier en zet mij even in de cc als je het opstuurt.
  • Lees je dit artikel over 3 jaar als de vacature verlopen is? Maakt vast niet uit, mail me maar.
  • Oh help, ik ben een beginner? Oh help, ben ik wel ervaren genoeg? Gewoon mailen danwel solliciteren. Ervaren met java/ruby/perl/APL in plaats van Python? Dan lukt Python vast ook wel. Mailen.
  • Niet geheel relevante vooropleiding? Ha ha ha. Onze systeembeheerder is afgestudeerd theoloog, de beste vormgever die we hebben is verpleger en we hebben ook een cardioloog als (goede!) programmeur. En ik heb bijvoorbeeld civiele techniek gestudeerd en niet technische informatica ofzo.
  • Utrecht is mooi maar je blijft liever dichter bij jouw huidige woonplaats in buurt? Ik vind het prima als je me mailt of ik nog ideeen heb, want ik krijg nogal eens de vraag van iemand "of ik nog iemand weet die...". Wil je bijvoorbeeld liever in de buurt van Rotterdam werken, kijk dan eens bij Zest software. Leuk bedrijf om bij te werken, ik heb er zelf 4 jaar gewerkt. En mijn broer Maurits werkt er nog steeds.
  • Heb ik al gezegd dat het leuk werken bij ons is? Kom eens langs of mail me :-)

24 Apr 2014 9:18pm GMT

Reinout van Rees: Python/django programmeren in hartje Utrecht (NL)

English note: I'm writing this in Dutch as it is for a job opening at our company. Call us old-fashioned, but we're all talking Dutch at the office :-)

We (= Nelen & Schuurmans) hebben weer versterking nodig van ons team Python programmeurs. Je kans om met Reinout te werken want de zaken gaan goed en er is veel werk te verstouwen :-) Laatst is een van onze ervaren programmeurs voor zichzelf begonnen en daarom gooi ik hier op mijn blog weer een hengeltje uit: we zijn specifiek ook op zoek naar een ervaren Python programmeur.

Ervaren? Nou, gewoon, een stukje kwaliteit en kunde en kennis inbrengen. Het mooiste compliment dat ik ooit van een collega (hoi Roland!) kreeg was dat 'ie mijn code altijd zo lekker leesbaar vond. Duidelijke variabelenamen, netjes gestructureerd, netjes pep8. Zoiets. Gewoon dat als je code schrijft dat het dan z'n werk goed doet en dat het niet afgeraffeld is. Dat je geheel vrijwillig de README.rst en de CHANGES.rst up to date houdt. En dat je na een maandje de eerste collega's aan je bureau krijgt met "kun je me even helpen met...". En dat je op een gegeven moment met ik-weet-niet-wat iets heel handigs hebt geautomatiseerd ofzo. Vanuit jezelf.

Grote bonuspunten als je veel met open source bezig bent geweest. Github, mailinglijsten, stackoverflow, enz. Veel van wat we als bedrijf doen is open source (vaak GPL, dan is het wel lekker open source maar kan tegelijkertijd de eventuele concurrent er niet zonder licentieproblemen een closed source slaatje uit slaan).

Het karakter van het bedrijf? We zijn geen puur IT bedrijf, het is echt een mix van IT en vakinhoud. Qua personele verhouding 1:2. Vakinhoudelijk is het waterveiligheid, waterkwaliteit, waterbeheer en vooral dus water. IT is een echte force multiplier voor ons. Wat we qua IT doen en kunnen versterkt de vakinhoudelijke kant enorm.

En we doen inmiddels best veel gave dingen. 3di is een compleet overstromingssimulatiepakket. Inclusief webinterface om het live aan te sturen. (Iemand heeft ook een video van het ontwikkelteam gemaakt). En kijk bijvoorbeeld naar http://regenradar.lizard.net/ (moet het wel net geregend hebben, zie anders deze video), da's op zich geen concurrent voor buienradar.nl maar geeft wel aan dat we neerslagdata hebben. En ook historische neerslagdata. En we integreren het met de rest van de data. Bij twee waterschappen berekenen we bijvoorbeeld aan de hand van neerslagvoorspellingen, meetgegevens, windrichting en getijde-voorspellingen hoe hard gemalen aangezet moeten worden. En dat gedetailleerde hoogtebestand dat laatst in het nieuws was (AHN2)? https://raster.lizard.net/wms/demo is een demo, we integreren het inmiddels in veel sites en we gebruiken het ook als achtergronddata voor dat 3Di dat ik eerder noemde.

Essentieel voor het karakter van het bedrijf is dat we allemaal hoog opgeleid zijn. Bijna iedereen universitair. En ik ben niet de enige dr. die er rondloopt: 8 van de 50 ofzo. Dat is eigenlijk de kern van het bedrijf: zet allemaal slimme, ijverige lui bij elkaar met een focus op waterveiligheid en er komen mooie (en winstgevende) dingen uit. Je hebt veel vrijheid (vertaald: het is soms een zooitje). Ruimte voor eigen initiatief.

Qua werkomgeving? Hartje Utrecht. Regelmatig halen we een ijsje bij dat ijskraampje op de Oude Gracht: dat is 1 minuut lopen. Goed bereikbaar: 10 minuten lopen vanaf het centraal station. Lunch enzo? Gezamenlijk in de kelder aan tafel. Vers brood van de bakker. Goed geregeld, wat dat betreft. Verder zijn veel dingen relatief goedkoop geregeld. Geen apple beeldschermen op het bureau zullen we maar zeggen. Voordeel: elk jaar hebben we tot nu toe winst gedraaid. Da's in deze tijd ook een geruststellende gedachte.

Werkinhoudelijk? Veel Python. Veel ook aan de web kant: Django. Er moet ook geautomatiseerd worden, dus Fabric, Ansible, buildout, enz. Her en der zit nog een snufje Java of Fortran. Natuurlijk ook veel javascript (recent ook steeds meer angularjs). Nette Python packages, changelog, readme, buildout, grunt/bower enz. Onze code staat op github. Branches, pull requests, code review. Tests kunnen we meer van gebruiken, maar we hebben ook al een aardige stapel. Zelfs javascript wordt deels getest. (Dus... qua tests kunnen we nog een extra opvoeder gebruiken). We hebben in twee datacenters een rack volgepropt, dus we hebben en vette VMWare omgeving met veel virtual machines. Best gaaf geregeld.

Het leukste: we hebben een groot IT team. Ongeveer 10 Python programmeurs (met bakken css/javascript/whatever kennis erbij). Da's binnen Nederland best een grote club. Volgens mij horen we gewoon bij de grootste Python bedrijven binnen Nederland. Je kan bij ons veel leren en sparren en experimenteren en helpen en samenwerken. Gaaf.

Interesse? Meer info nodig? Je kan kijken naar alles wat ik met "nelenschuurmans" getagt heb op dit weblog. Je kan mij mailen op reinout@vanrees.org.

  • De officiele vacature staat hier: http://www.nelen-schuurmans.nl/NL/bedrijfsprofiel/carriere . Het emailadres voor de sollicitatie staat daar ook bij. Doe me een plezier en zet mij even in de cc als je het opstuurt.
  • Lees je dit artikel over 3 jaar als de vacature verlopen is? Maakt vast niet uit, mail me maar.
  • Oh help, ik ben een beginner? Oh help, ben ik wel ervaren genoeg? Gewoon mailen danwel solliciteren. Ervaren met java/ruby/perl/APL in plaats van Python? Dan lukt Python vast ook wel. Mailen.
  • Niet geheel relevante vooropleiding? Ha ha ha. Onze systeembeheerder is afgestudeerd theoloog, de beste vormgever die we hebben is verpleger en we hebben ook een cardioloog als (goede!) programmeur. En ik heb bijvoorbeeld civiele techniek gestudeerd en niet technische informatica ofzo.
  • Utrecht is mooi maar je blijft liever dichter bij jouw huidige woonplaats in buurt? Ik vind het prima als je me mailt of ik nog ideeen heb, want ik krijg nogal eens de vraag van iemand "of ik nog iemand weet die...". Wil je bijvoorbeeld liever in de buurt van Rotterdam werken, kijk dan eens bij Zest software. Leuk bedrijf om bij te werken, ik heb er zelf 4 jaar gewerkt. En mijn broer Maurits werkt er nog steeds.
  • Heb ik al gezegd dat het leuk werken bij ons is? Kom eens langs of mail me :-)

24 Apr 2014 9:18pm GMT

Reinout van Rees: Python/django programmeren in hartje Utrecht (NL)

English note: I'm writing this in Dutch as it is for a job opening at our company. Call us old-fashioned, but we're all talking Dutch at the office :-)

We (= Nelen & Schuurmans) hebben weer versterking nodig van ons team Python programmeurs. Je kans om met Reinout te werken want de zaken gaan goed en er is veel werk te verstouwen. Laatst is een van onze ervaren programmeurs voor zichzelf begonnen en daarom gooi ik hier op mijn blog weer een hengeltje uit: we zijn specifiek ook op zoek naar een ervaren Python programmeur.

Ervaren? Nou, gewoon, een stukje kwaliteit en kunde en kennis inbrengen. Het mooiste compliment dat ik ooit van een collega (hoi Roland!) kreeg was dat 'ie mijn code altijd zo lekker leesbaar vond. Duidelijke variabelenamen, netjes gestructureerd, netjes pep8. Zoiets. Gewoon dat als je code schrijft dat het dan z'n werk goed doet en dat het niet afgeraffeld is. Dat je geheel vrijwillig de README.rst en de CHANGES.rst up to date houdt. En dat je na een maandje de eerste collega's aan je bureau krijgt met "kun je me even helpen met...". En dat je op een gegeven moment met ik-weet-niet-wat iets heel handigs hebt geautomatiseerd ofzo. Vanuit jezelf.

Grote bonuspunten als je veel met open source bezig bent geweest. Github, mailinglijsten, stackoverflow, enz. Veel van wat we als bedrijf doen is ook open source (vaak GPL, dan is het wel lekker open source maar kan tegelijkertijd de eventuele concurrent er niet zonder licentieproblemen een closed source slaatje uit slaan).

Het karakter van het bedrijf? We zijn geen puur IT bedrijf, het is echt een mix van IT en vakinhoud. Qua personele verhouding 1:2. Vakinhoudelijk is het waterveiligheid, waterkwaliteit, waterbeheer en vooral dus water. IT is een echte force multiplier voor ons. Wat we qua IT doen en kunnen versterkt de vakinhoudelijke kant enorm.

En we doen inmiddels best veel gave dingen. 3di is een compleet overstromingssimulatiepakket. Inclusief webinterface om het live aan te sturen. (Iemand heeft ook een video van het ontwikkelteam gemaakt) En kijk bijvoorbeeld naar http://regenradar.lizard.net/ (moet het wel net geregend hebben, zie anders deze video), da's op zich geen concurrent voor buienradar.nl maar geeft wel aan dat we neerslagdata hebben. En ook historische neerslagdata. En we integreren het met de rest van de data. Bij twee waterschappen berekenen we bijvoorbeeld aan de hand van neerslagvoorspellingen, meetgegevens, windrichting en getijde-voorspellingen hoe hard gemalen aangezet moeten worden. En dat gedetailleerde hoogtebestand dat laatst in het nieuws was (AHN2)? https://raster.lizard.net/wms/demo is een demo, we integreren het inmiddels in veel sites en we gebruiken het ook als achtergronddata voo dat 3Di dat ik eerder noemde.

Essentieel voor het karakter van het bedrijf is dat we allemaal hoog opgeleid zijn. Bijna iedereen universitair. En ik ben niet de enige dr.ir. die er rondloopt: 8 van de 50 ofzo. Dat is eigenlijk de kern van het bedrijf: zet allemaal slimme, ijverige lui bij elkaar met een focus op waterveiligheid en er komen mooie (en winstgevende) dingen uit. Je hebt veel vrijheid (vertaald: het is soms een zooitje). Ruimte voor eigen initiatief.

Qua werkomgeving? Hartje Utrecht. Regelmatig halen we een ijsje bij dat ijskraampje op de Oude Gracht: dat is 1 minuut lopen. Goed bereikbaar: 10 minuten lopen vanaf het centraal station. Lunch enzo? Gezamenlijk in de kelder aan tafels. Vers brood van de bakker. Goed geregeld, wat dat betreft. Verder zijn veel dingen relatief goedkoop geregeld. Geen apple beeldschermen op het bureau zullen we maar zeggen. Voordeel: elk jaar hebben we tot nu toe winst gedraaid. Da's in deze tijd ook een geruststellende gedachte.

Werkinhoudelijk? Veel Python. Veel ook aan de web kant: Django. Er moet ook geautomatiseerd worden, dus Fabric, Ansible, buildout, enz. Her en der zit nog een snufje Java of Fortran. Natuurlijk ook vrij veel javascript; we gebruiken ook aardig wat angularjs. Nette Python packages, changelog, readme, buildout, grunt/bower enz. Onze code staat op github. Branches, pull requests, code review. Tests kunnen we meer van gebruiken, maar we hebben ook al een aardige stapel. Zelfs javascript wordt deels getest. (Dus... qua tests kunnen we nog een extra opvoeder gebruiken).

Het leukste: we hebben een groot IT team. Ongeveer 10 Python programmeurs (met bakken css/javascript/whatever kennis erbij). Da's binnen Nederland best een grote club. Volgens mij horen we gewoon bij de grootste Python bedrijven binnen Nederland. Je kan bij ons veel leren en sparren en experimenteren en helpen en samenwerken. Gaaf.

Interesse? Meer info nodig? Je kan kijken naar alles wat ik met nelenschuurmans getagt heb op dit weblog. Je kan mij ook mailen op reinout@vanrees.org.

  • De officiele vacature staat hier: http://www.nelen-schuurmans.nl/NL/bedrijfsprofiel/carriere . Het emailadres voor de sollicitatie staat daar ook bij. Doe me een plezier en zet mij even in de CC :-)
  • Lees je dit artikel over 3 jaar als de vacature verlopen is? Maakt vast niet uit, mail me maar.
  • Oh help, ik ben een beginner? Oh help, ben ik wel ervaren genoeg? Gewoon mailen danwel solliciteren. Ervaren met java/ruby/perl/APL? Dan lukt Python vast ook wel. Mailen.
  • Utrecht is mooi maar je blijft liever dichter in jouw buurt? Ik vind het prima als je me mailt of ik nog ideeen heb, want ik krijg nogal eens de vraag van iemand "of ik nog iemand weet die...". Wil je bijvoorbeeld liever in de buurt van Rotterdam werken, kijk dan eens bij Zest software bijvoorbeeld. Leuk bedrijf om bij te werken, ik heb er zelf 4 jaar gewerkt.
  • Heb ik al gezegd dat het leuk werken bij ons is? Kom eens langs of mail me :-)

24 Apr 2014 8:54pm GMT

Reinout van Rees: Python/django programmeren in hartje Utrecht (NL)

English note: I'm writing this in Dutch as it is for a job opening at our company. Call us old-fashioned, but we're all talking Dutch at the office :-)

We (= Nelen & Schuurmans) hebben weer versterking nodig van ons team Python programmeurs. Je kans om met Reinout te werken want de zaken gaan goed en er is veel werk te verstouwen. Laatst is een van onze ervaren programmeurs voor zichzelf begonnen en daarom gooi ik hier op mijn blog weer een hengeltje uit: we zijn specifiek ook op zoek naar een ervaren Python programmeur.

Ervaren? Nou, gewoon, een stukje kwaliteit en kunde en kennis inbrengen. Het mooiste compliment dat ik ooit van een collega (hoi Roland!) kreeg was dat 'ie mijn code altijd zo lekker leesbaar vond. Duidelijke variabelenamen, netjes gestructureerd, netjes pep8. Zoiets. Gewoon dat als je code schrijft dat het dan z'n werk goed doet en dat het niet afgeraffeld is. Dat je geheel vrijwillig de README.rst en de CHANGES.rst up to date houdt. En dat je na een maandje de eerste collega's aan je bureau krijgt met "kun je me even helpen met...". En dat je op een gegeven moment met ik-weet-niet-wat iets heel handigs hebt geautomatiseerd ofzo. Vanuit jezelf.

Grote bonuspunten als je veel met open source bezig bent geweest. Github, mailinglijsten, stackoverflow, enz. Veel van wat we als bedrijf doen is ook open source (vaak GPL, dan is het wel lekker open source maar kan tegelijkertijd de eventuele concurrent er niet zonder licentieproblemen een closed source slaatje uit slaan).

Het karakter van het bedrijf? We zijn geen puur IT bedrijf, het is echt een mix van IT en vakinhoud. Qua personele verhouding 1:2. Vakinhoudelijk is het waterveiligheid, waterkwaliteit, waterbeheer en vooral dus water. IT is een echte force multiplier voor ons. Wat we qua IT doen en kunnen versterkt de vakinhoudelijke kant enorm.

En we doen inmiddels best veel gave dingen. 3di is een compleet overstromingssimulatiepakket. Inclusief webinterface om het live aan te sturen. (Iemand heeft ook een video van het ontwikkelteam gemaakt) En kijk bijvoorbeeld naar http://regenradar.lizard.net/ (moet het wel net geregend hebben, zie anders deze video), da's op zich geen concurrent voor buienradar.nl maar geeft wel aan dat we neerslagdata hebben. En ook historische neerslagdata. En we integreren het met de rest van de data. Bij twee waterschappen berekenen we bijvoorbeeld aan de hand van neerslagvoorspellingen, meetgegevens, windrichting en getijde-voorspellingen hoe hard gemalen aangezet moeten worden. En dat gedetailleerde hoogtebestand dat laatst in het nieuws was (AHN2)? https://raster.lizard.net/wms/demo is een demo, we integreren het inmiddels in veel sites en we gebruiken het ook als achtergronddata voo dat 3Di dat ik eerder noemde.

Essentieel voor het karakter van het bedrijf is dat we allemaal hoog opgeleid zijn. Bijna iedereen universitair. En ik ben niet de enige dr.ir. die er rondloopt: 8 van de 50 ofzo. Dat is eigenlijk de kern van het bedrijf: zet allemaal slimme, ijverige lui bij elkaar met een focus op waterveiligheid en er komen mooie (en winstgevende) dingen uit. Je hebt veel vrijheid (vertaald: het is soms een zooitje). Ruimte voor eigen initiatief.

Qua werkomgeving? Hartje Utrecht. Regelmatig halen we een ijsje bij dat ijskraampje op de Oude Gracht: dat is 1 minuut lopen. Goed bereikbaar: 10 minuten lopen vanaf het centraal station. Lunch enzo? Gezamenlijk in de kelder aan tafels. Vers brood van de bakker. Goed geregeld, wat dat betreft. Verder zijn veel dingen relatief goedkoop geregeld. Geen apple beeldschermen op het bureau zullen we maar zeggen. Voordeel: elk jaar hebben we tot nu toe winst gedraaid. Da's in deze tijd ook een geruststellende gedachte.

Werkinhoudelijk? Veel Python. Veel ook aan de web kant: Django. Er moet ook geautomatiseerd worden, dus Fabric, Ansible, buildout, enz. Her en der zit nog een snufje Java of Fortran. Natuurlijk ook vrij veel javascript; we gebruiken ook aardig wat angularjs. Nette Python packages, changelog, readme, buildout, grunt/bower enz. Onze code staat op github. Branches, pull requests, code review. Tests kunnen we meer van gebruiken, maar we hebben ook al een aardige stapel. Zelfs javascript wordt deels getest. (Dus... qua tests kunnen we nog een extra opvoeder gebruiken).

Het leukste: we hebben een groot IT team. Ongeveer 10 Python programmeurs (met bakken css/javascript/whatever kennis erbij). Da's binnen Nederland best een grote club. Volgens mij horen we gewoon bij de grootste Python bedrijven binnen Nederland. Je kan bij ons veel leren en sparren en experimenteren en helpen en samenwerken. Gaaf.

Interesse? Meer info nodig? Je kan kijken naar alles wat ik met nelenschuurmans getagt heb op dit weblog. Je kan mij ook mailen op reinout@vanrees.org.

  • De officiele vacature staat hier: http://www.nelen-schuurmans.nl/NL/bedrijfsprofiel/carriere . Het emailadres voor de sollicitatie staat daar ook bij. Doe me een plezier en zet mij even in de CC :-)
  • Lees je dit artikel over 3 jaar als de vacature verlopen is? Maakt vast niet uit, mail me maar.
  • Oh help, ik ben een beginner? Oh help, ben ik wel ervaren genoeg? Gewoon mailen danwel solliciteren. Ervaren met java/ruby/perl/APL? Dan lukt Python vast ook wel. Mailen.
  • Utrecht is mooi maar je blijft liever dichter in jouw buurt? Ik vind het prima als je me mailt of ik nog ideeen heb, want ik krijg nogal eens de vraag van iemand "of ik nog iemand weet die...". Wil je bijvoorbeeld liever in de buurt van Rotterdam werken, kijk dan eens bij Zest software bijvoorbeeld. Leuk bedrijf om bij te werken, ik heb er zelf 4 jaar gewerkt.
  • Heb ik al gezegd dat het leuk werken bij ons is? Kom eens langs of mail me :-)

24 Apr 2014 8:54pm GMT

Kay Hayen: Try Finally Python Quiz

When working on my Python compiler Nuitka, I often come across ridiculous language details of the Python language, and turn these into quizzes, for which I finally added a dedicated quiz tag.

Anyway, who can predict, what these will do to you:

def f():
    try:
        return 1
    finally:
        return 2

Will it return 1 or 2 ?

def f():
    try:
        1/0
    finally:
        return 2

Will this raise an ZeroDivisionError or return 2 ?

def f():
    while 1
        try:
            continue
        finally:
            break

Is this an endless loop or does it return?

def f():
    while 1
        try:
            break
        finally:
            continue

What about that? This one holds an inconsistency.

No solutions yet this time.

24 Apr 2014 8:08pm GMT

Kay Hayen: Try Finally Python Quiz

When working on my Python compiler Nuitka, I often come across ridiculous language details of the Python language, and turn these into quizzes, for which I finally added a dedicated quiz tag.

Anyway, who can predict, what these will do to you:

def f():
    try:
        return 1
    finally:
        return 2

Will it return 1 or 2 ?

def f():
    try:
        1/0
    finally:
        return 2

Will this raise an ZeroDivisionError or return 2 ?

def f():
    while 1
        try:
            continue
        finally:
            break

Is this an endless loop or does it return?

def f():
    while 1
        try:
            break
        finally:
            continue

What about that? This one holds an inconsistency.

No solutions yet this time.

24 Apr 2014 8:08pm GMT

Martijn Faassen: Morepath 0.2

I've just released Morepath 0.2! Morepath is your friendly neighborhood Python web microframework with super powers.

Major changes include:

And various smaller tweaks. You can also read the changelog.

What makes Morepath special? You can read about its superpowers and read how it compares to other web frameworks, but here are some highlights:

It's lots of power contained in a small codebase.

(If you followed the Morepath link above, ignore the 0.1 in the title bar there: those are the 0.2 docs, not 0.1. I've now fixed the doc building configuration to include the version number from the package itself.)

24 Apr 2014 2:38pm GMT

Martijn Faassen: Morepath 0.2

I've just released Morepath 0.2! Morepath is your friendly neighborhood Python web microframework with super powers.

Major changes include:

And various smaller tweaks. You can also read the changelog.

What makes Morepath special? You can read about its superpowers and read how it compares to other web frameworks, but here are some highlights:

It's lots of power contained in a small codebase.

(If you followed the Morepath link above, ignore the 0.1 in the title bar there: those are the 0.2 docs, not 0.1. I've now fixed the doc building configuration to include the version number from the package itself.)

24 Apr 2014 2:38pm GMT

eGenix.com: Python Meeting Düsseldorf - 2014-04-29

The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.

Ankündigung

Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:

Dienstag, 24.09.2014, 19:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf


Neuigkeiten

Bereits angemeldete Vorträge

Charlie Clark
"Status openpyxl, bzw. Lösung neuer Probleme"
"IndexedList - eine Liste optimiert für "in" Abfragen"
"Bericht von der PyCon 2014 in Montreal"


Marc-Andre Lemburg
"Python Code mit lib2to3 modernisieren"
"DDOS Attacken mit Python bekämpfen"
"Bericht von der FOSDEM 2014"

Wir suchen noch weitere Vorträge. Bei Interesse, bitte unter info@pyddf.de melden.

Geänderte Startzeit

Dieses Mal treffen wir uns erst um 19:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden, da wir keinen Termin für 18 Uhr bekommen haben. Hier eine kurze Beschreibung:

Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.

Über dem Eingang steht ein großes "Schwimm'in Bilk" Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.

>>> Eingang in Google Street View

Einleitung

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

Programm

Das Python Meeting Düsseldorf nutzt eine Mischung aus Open Space und Lightning Talks, wobei die Gewitter bei uns auch schon mal 20 Minuten dauern können :-)

Lightning Talks können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit XGA Auflösung steht zur Verfügung. Folien bitte als PDF auf USB Stick mitbringen.

Lightning Talk Anmeldung bitte formlos per EMail an info@pyddf.de

Kostenbeteiligung

Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.

Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.

Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.

Anmeldung

Da wir nur für ca. 20 Personen Sitzplätze haben, möchten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung.

Meeting Anmeldung bitte formlos per EMail an info@pyddf.de

Weitere Informationen

Weitere Informationen finden Sie auf der Webseite des Meetings:

http://pyddf.de/

Viel Spaß !

Marc-Andre Lemburg, eGenix.com

Published: 2013-03-20
Please enable JavaScript to make full use of our web-site. Thank you.
Contact : Impressum : Terms & Conditions : Privacy Policy : Trademarks 2013-03-20
(c) Copyright 2000-2013 eGenix.com Software, Skills and Services GmbH, Langenfeld, Germany. All Rights Reserved.

24 Apr 2014 9:00am GMT

eGenix.com: Python Meeting Düsseldorf - 2014-04-29

The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.

Ankündigung

Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:

Dienstag, 24.09.2014, 19:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf


Neuigkeiten

Bereits angemeldete Vorträge

Charlie Clark
"Status openpyxl, bzw. Lösung neuer Probleme"
"IndexedList - eine Liste optimiert für "in" Abfragen"
"Bericht von der PyCon 2014 in Montreal"


Marc-Andre Lemburg
"Python Code mit lib2to3 modernisieren"
"DDOS Attacken mit Python bekämpfen"
"Bericht von der FOSDEM 2014"

Wir suchen noch weitere Vorträge. Bei Interesse, bitte unter info@pyddf.de melden.

Geänderte Startzeit

Dieses Mal treffen wir uns erst um 19:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden, da wir keinen Termin für 18 Uhr bekommen haben. Hier eine kurze Beschreibung:

Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.

Über dem Eingang steht ein großes "Schwimm'in Bilk" Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.

>>> Eingang in Google Street View

Einleitung

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

Programm

Das Python Meeting Düsseldorf nutzt eine Mischung aus Open Space und Lightning Talks, wobei die Gewitter bei uns auch schon mal 20 Minuten dauern können :-)

Lightning Talks können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit XGA Auflösung steht zur Verfügung. Folien bitte als PDF auf USB Stick mitbringen.

Lightning Talk Anmeldung bitte formlos per EMail an info@pyddf.de

Kostenbeteiligung

Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.

Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.

Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.

Anmeldung

Da wir nur für ca. 20 Personen Sitzplätze haben, möchten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung.

Meeting Anmeldung bitte formlos per EMail an info@pyddf.de

Weitere Informationen

Weitere Informationen finden Sie auf der Webseite des Meetings:

http://pyddf.de/

Viel Spaß !

Marc-Andre Lemburg, eGenix.com

Published: 2013-03-20
Please enable JavaScript to make full use of our web-site. Thank you.
Contact : Impressum : Terms & Conditions : Privacy Policy : Trademarks 2013-03-20
(c) Copyright 2000-2013 eGenix.com Software, Skills and Services GmbH, Langenfeld, Germany. All Rights Reserved.

24 Apr 2014 9:00am GMT

Alec Munro: PyCon 2014

I've now been back from PyCon for a week, and I've got some thoughts to share.

Scope

It was huge.

I usually try to memorize everyone's names, and I have some habits that help me with that. But there were so many people, I think that may have fallen apart. :)

A lot of hero worship, as I met, or at least observed from a distance, many people who helped shape my views on software (+Tres Seaver in particular).

Conversely, I managed to avoid running into those attending from my employers (I'm looking at you, +Kenneth Reitz, Sandy Walsh, and probably someone from RIM/BlackBerry).

Diversity

All the promotion of the diversity was terrific. At the same time that it's great to be part of a movement that is markedly more female-friendly then the tech community at large, Jessica McKellar made it clear that we have so much farther to go. As the father of two girls, it's very important to me that we change the culture around technology to emphasize that there's no particular skillset or aptitude that's required for entry.

Software is our world, and we can empower EVERYONE to play a part in shaping it.

Content Overview

I enjoyed the talks that I went to, but I did skip more than I was intending to. I had trouble letting go of work, and there was a lot of content that was relatively beginner focused, or represented tutorials that I knew had high-quality online counterparts, should I need them. I feel like this was a deficiency of my own, and one I hope I handle better if I come back next year.

Meta programming

I've been flirting with creating my own language for a while now, and if I were to do so, it would probably be on top of Python. Thanks to talks by +Allison Kaptur and +Paul Tagliamonte, I feel much more prepared to do so.

Allison provided a brilliant guide to implementing the import mechanism from scratch. Having read +Brett Cannon's blog when he created importlib, I knew there was a huge amount of work that went into getting it right, so it was an intimidating area. Yet in 20 minutes Allison walked us through getting something functional.

Paul's talk on Hy was not quite so accessible, but perhaps even more inspiring. The relative ease with which Hy and Python can co-exist within the same project is just awesome, though mucking around with ASTs remains a bit of a scary idea.

Sprints

While I was skipping talks, I consoled myself in the thought that I would really engage during the Sprints (I had a day and half scheduled for these). But I didn't, and while I think that had more to do with me (once again, I worked), I'll share what I think could have been done better, in case anyone else felt the same way.

Technically Sprints started Sunday evening, but I get the feeling that no one was actually interested in running them Sunday evening (or maybe my timing was off). There were a handful of people there, but no clear organization or plan about what was to be worked on.

Monday morning, it was certainly better attended, but it still wasn't inviting. There was a central chart of what rooms contained what projects, but within the rooms there was no indication of who was working on what. From my limited time being involved in or running other short coding sessions, I was also surprised not to see much use of flipcharts or whiteboards.

I guess how I would do it, if I ran it next year (I'm not volunteering, yet), is provide signs for each project to put at their table, and encourage each of them to write out a list of goals for the sprint in a place where it can be publicly examined and crossed off. Perhaps also provide special shirts/hats/badges to the "leader" of each sprint. The experience I would like is for someone to be able to wander in, examine what each project is doing without actually bothering anybody, and then if they find something they think could fit them, to know who to ask.

Misc.


Conclusion

As someone who has been around the block but doesn't find much time to actually code anymore, I may not be the code audience for PyCon. But I'm still delighted to have finally made it to one, and I'm really tempted to make it a family trip next year.

24 Apr 2014 8:43am GMT

Alec Munro: PyCon 2014

I've now been back from PyCon for a week, and I've got some thoughts to share.

Scope

It was huge.

I usually try to memorize everyone's names, and I have some habits that help me with that. But there were so many people, I think that may have fallen apart. :)

A lot of hero worship, as I met, or at least observed from a distance, many people who helped shape my views on software (+Tres Seaver in particular).

Conversely, I managed to avoid running into those attending from my employers (I'm looking at you, +Kenneth Reitz, Sandy Walsh, and probably someone from RIM/BlackBerry).

Diversity

All the promotion of the diversity was terrific. At the same time that it's great to be part of a movement that is markedly more female-friendly then the tech community at large, Jessica McKellar made it clear that we have so much farther to go. As the father of two girls, it's very important to me that we change the culture around technology to emphasize that there's no particular skillset or aptitude that's required for entry.

Software is our world, and we can empower EVERYONE to play a part in shaping it.

Content Overview

I enjoyed the talks that I went to, but I did skip more than I was intending to. I had trouble letting go of work, and there was a lot of content that was relatively beginner focused, or represented tutorials that I knew had high-quality online counterparts, should I need them. I feel like this was a deficiency of my own, and one I hope I handle better if I come back next year.

Meta programming

I've been flirting with creating my own language for a while now, and if I were to do so, it would probably be on top of Python. Thanks to talks by +Allison Kaptur and +Paul Tagliamonte, I feel much more prepared to do so.

Allison provided a brilliant guide to implementing the import mechanism from scratch. Having read +Brett Cannon's blog when he created importlib, I knew there was a huge amount of work that went into getting it right, so it was an intimidating area. Yet in 20 minutes Allison walked us through getting something functional.

Paul's talk on Hy was not quite so accessible, but perhaps even more inspiring. The relative ease with which Hy and Python can co-exist within the same project is just awesome, though mucking around with ASTs remains a bit of a scary idea.

Sprints

While I was skipping talks, I consoled myself in the thought that I would really engage during the Sprints (I had a day and half scheduled for these). But I didn't, and while I think that had more to do with me (once again, I worked), I'll share what I think could have been done better, in case anyone else felt the same way.

Technically Sprints started Sunday evening, but I get the feeling that no one was actually interested in running them Sunday evening (or maybe my timing was off). There were a handful of people there, but no clear organization or plan about what was to be worked on.

Monday morning, it was certainly better attended, but it still wasn't inviting. There was a central chart of what rooms contained what projects, but within the rooms there was no indication of who was working on what. From my limited time being involved in or running other short coding sessions, I was also surprised not to see much use of flipcharts or whiteboards.

I guess how I would do it, if I ran it next year (I'm not volunteering, yet), is provide signs for each project to put at their table, and encourage each of them to write out a list of goals for the sprint in a place where it can be publicly examined and crossed off. Perhaps also provide special shirts/hats/badges to the "leader" of each sprint. The experience I would like is for someone to be able to wander in, examine what each project is doing without actually bothering anybody, and then if they find something they think could fit them, to know who to ask.

Misc.


Conclusion

As someone who has been around the block but doesn't find much time to actually code anymore, I may not be the code audience for PyCon. But I'm still delighted to have finally made it to one, and I'm really tempted to make it a family trip next year.

24 Apr 2014 8:43am GMT

eGenix.com: eGenix mxODBC Connect 2.0.5 GA

Introduction

The mxODBC Connect Database Interface for Python allows users to easily connect Python applications to all major databases on the market today in a highly portable, convenient and secure way.

Python Database Connectivity the Easy Way

Unlike our mxODBC Python extension, mxODBC Connect is designed as client-server application, so you no longer need to find production quality ODBC drivers for all the platforms you target with your Python application.

Instead you use an easy to install royalty-free Python client library which connects directly to the mxODBC Connect database server over the network.

This makes mxODBC Connect a great basis for writing cross-platform multi-tier database applications and utilities in Python, especially if you run applications that need to communicate with databases such as MS SQL Server and MS Access, Oracle Database, IBM DB2 and Informix, Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more, that run on Windows or Linux machines.

Ideal for Database Driven Client Applications

By removing the need to install and configure ODBC drivers on the client side and dealing with complicated network setups for each set of drivers, mxODBC Connect greatly simplifies deployment of database driven client applications, while at the same time making the network communication between client and database server more efficient and more secure.

For more information, please have a look at the mxODBC Connect product page, in particular, the full list of available features.

News

The 2.0.5 release of mxODBC Connect includes the following enhancements and fixes:

Security Enhancements

mxODBC Connect 2.0 was released on 2012-08-20. These are the highlights of the new release:

Enhanced API

Updated Compatibility

Asynchronous Execution

Better Integration

For the full set of changes, please check the mxODBC Connect change log.

Upgrading

You are encouraged to upgrade to this latest mxODBC Connect release. When upgrading, please always upgrade both the server and the client installations to the same version - even for patch level releases.

Customers who have purchased mxODBC Connect 2.0 licenses can continue to use their licenses with this patch level release.

Customers who have purchased mxODBC Connect 1.x licenses can request 20% discount coupons for upgrade purchases. Please contact the eGenix.com Sales Team with your existing license serials for details.

Users of our stand-alone mxODBC product will have to purchase new licenses from our online shop in order to use mxODBC Connect.

You can request 30-day evaluation licenses via our web-site or writing to sales@egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need.

Downloads

Please visit the eGenix mxODBC Connect product page for downloads, instructions on installation and documentation of the client and the server package.

If you want to try the package, please jump straight to the download instructions.

Fully functional evaluation licenses for the mxODBC Connect Server are available free of charge.

mxODBC Connect Client is always free of charge.

Support

Commercial support for this product is available directly from eGenix.com.

Please see the support section of our website for details.

More Information

For more information on the eGenix mxODBC Connect, licensing and download instructions, please write to sales@egenix.com.

Enjoy !

Marc-Andre Lemburg, eGenix.com

24 Apr 2014 8:00am GMT

eGenix.com: eGenix mxODBC Connect 2.0.5 GA

Introduction

The mxODBC Connect Database Interface for Python allows users to easily connect Python applications to all major databases on the market today in a highly portable, convenient and secure way.

Python Database Connectivity the Easy Way

Unlike our mxODBC Python extension, mxODBC Connect is designed as client-server application, so you no longer need to find production quality ODBC drivers for all the platforms you target with your Python application.

Instead you use an easy to install royalty-free Python client library which connects directly to the mxODBC Connect database server over the network.

This makes mxODBC Connect a great basis for writing cross-platform multi-tier database applications and utilities in Python, especially if you run applications that need to communicate with databases such as MS SQL Server and MS Access, Oracle Database, IBM DB2 and Informix, Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more, that run on Windows or Linux machines.

Ideal for Database Driven Client Applications

By removing the need to install and configure ODBC drivers on the client side and dealing with complicated network setups for each set of drivers, mxODBC Connect greatly simplifies deployment of database driven client applications, while at the same time making the network communication between client and database server more efficient and more secure.

For more information, please have a look at the mxODBC Connect product page, in particular, the full list of available features.

News

The 2.0.5 release of mxODBC Connect includes the following enhancements and fixes:

Security Enhancements

mxODBC Connect 2.0 was released on 2012-08-20. These are the highlights of the new release:

Enhanced API

Updated Compatibility

Asynchronous Execution

Better Integration

For the full set of changes, please check the mxODBC Connect change log.

Upgrading

You are encouraged to upgrade to this latest mxODBC Connect release. When upgrading, please always upgrade both the server and the client installations to the same version - even for patch level releases.

Customers who have purchased mxODBC Connect 2.0 licenses can continue to use their licenses with this patch level release.

Customers who have purchased mxODBC Connect 1.x licenses can request 20% discount coupons for upgrade purchases. Please contact the eGenix.com Sales Team with your existing license serials for details.

Users of our stand-alone mxODBC product will have to purchase new licenses from our online shop in order to use mxODBC Connect.

You can request 30-day evaluation licenses via our web-site or writing to sales@egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need.

Downloads

Please visit the eGenix mxODBC Connect product page for downloads, instructions on installation and documentation of the client and the server package.

If you want to try the package, please jump straight to the download instructions.

Fully functional evaluation licenses for the mxODBC Connect Server are available free of charge.

mxODBC Connect Client is always free of charge.

Support

Commercial support for this product is available directly from eGenix.com.

Please see the support section of our website for details.

More Information

For more information on the eGenix mxODBC Connect, licensing and download instructions, please write to sales@egenix.com.

Enjoy !

Marc-Andre Lemburg, eGenix.com

24 Apr 2014 8:00am GMT

Wingware: Wing IDE 5.0.6 Released

Wingware has released version 5.0.6 of Wing IDE, our cross-platform integrated
development environment for the Python programming language.

Wing IDE includes a professional quality code editor with vi, emacs, visual studio,
and other key bindings, auto-completion, call tips, goto-definition, find uses, refactoring,
context-aware auto-editing, a powerful graphical debugger, version control, unit testing,
search, and many other features. For details see http://wingware.com/

Changes in this minor release include:

For details see http://wingware.com/pub/wingide/5.0.6/CHANGELOG.txt

A summary of new features in Wing 5:

For more information on what's new in Wing 5, see http://wingware.com/wingide/whatsnew

Free trial: http://wingware.com/wingide/trial
Downloads: http://wingware.com/downloads
Feature list: http://wingware.com/wingide/features
Sales: http://wingware.com/store/purchase
Upgrades: https://wingware.com/store/upgrade

Questions? Don't hesitate to email us at support@wingware.com.

24 Apr 2014 7:06am GMT

Wingware: Wing IDE 5.0.6 Released

Wingware has released version 5.0.6 of Wing IDE, our cross-platform integrated
development environment for the Python programming language.

Wing IDE includes a professional quality code editor with vi, emacs, visual studio,
and other key bindings, auto-completion, call tips, goto-definition, find uses, refactoring,
context-aware auto-editing, a powerful graphical debugger, version control, unit testing,
search, and many other features. For details see http://wingware.com/

Changes in this minor release include:

For details see http://wingware.com/pub/wingide/5.0.6/CHANGELOG.txt

A summary of new features in Wing 5:

For more information on what's new in Wing 5, see http://wingware.com/wingide/whatsnew

Free trial: http://wingware.com/wingide/trial
Downloads: http://wingware.com/downloads
Feature list: http://wingware.com/wingide/features
Sales: http://wingware.com/store/purchase
Upgrades: https://wingware.com/store/upgrade

Questions? Don't hesitate to email us at support@wingware.com.

24 Apr 2014 7:06am GMT

Richard Jones: PyCon AU 2014 CFP about to close!

The PyCon Australia 2014 CFP is about to close! Last chance to get your proposal in! Quoted:

The conference this year will be held on Saturday 2 and Sunday 3 August 2014 in Brisbane. We'll also be featuring a day of miniconfs on Friday 1 August.

The deadline for proposal submission is Friday 25 April, 2014.

PyCon Australia attracts professional developers from all walks of life, including industry, government, and science, as well as enthusiast and student developers. We're looking for proposals for presentations and tutorials on any aspect of Python programming, at all skill levels from novice to advanced.

Presentation subjects may range from reports on open source, academic or commercial projects; or even tutorials and case studies. If a presentation is interesting and useful to the Python community, it will be considered for inclusion in the program.

We're especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?

Proposals about the Django web framework are very strongly encouraged, and will also be considered for inclusion in DjangoCon AU, to be held on Friday 1 August.

There will also be a Science and Data Analysis miniconf and an OpenStack miniconf held alongside DjangoCon AU. Proposals on either of these topics will be considered for inclusion in these miniconfs.

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

See you in Brisbane in August!

24 Apr 2014 5:31am GMT

Richard Jones: PyCon AU 2014 CFP about to close!

The PyCon Australia 2014 CFP is about to close! Last chance to get your proposal in! Quoted:

The conference this year will be held on Saturday 2 and Sunday 3 August 2014 in Brisbane. We'll also be featuring a day of miniconfs on Friday 1 August.

The deadline for proposal submission is Friday 25 April, 2014.

PyCon Australia attracts professional developers from all walks of life, including industry, government, and science, as well as enthusiast and student developers. We're looking for proposals for presentations and tutorials on any aspect of Python programming, at all skill levels from novice to advanced.

Presentation subjects may range from reports on open source, academic or commercial projects; or even tutorials and case studies. If a presentation is interesting and useful to the Python community, it will be considered for inclusion in the program.

We're especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?

Proposals about the Django web framework are very strongly encouraged, and will also be considered for inclusion in DjangoCon AU, to be held on Friday 1 August.

There will also be a Science and Data Analysis miniconf and an OpenStack miniconf held alongside DjangoCon AU. Proposals on either of these topics will be considered for inclusion in these miniconfs.

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

See you in Brisbane in August!

24 Apr 2014 5:31am GMT

Filipe Saraiva: KDE packagers: give some love to Cantor

python2_select

I have some posts to write about Cantor but first I would like to request a help to KDE packagers of several Linux distros around the world.

I received some mails from users asking "how can I use python in Cantor?" or "where is python support in Cantor?". Well, python2-backend is available in Cantor since KDE 4.12 release. If you is using KDE >= 4.12 but you can not to use python in Cantor, maybe the package was not build correctly.

python 2 development library (commonly packed as python-devel in some Linux distros) is required to build python2-backend. python 2 is required to use Cantor with python 2.

Then if you are a Cantor user and can not to use Cantor with python, please write a bug report in the bug management system of your distro. You can to put a link in the bug report to this post too.

Anyway, if your distro bring or not bring python2-backend, write a comment below and I will make a table with this information.

24 Apr 2014 12:47am GMT

Filipe Saraiva: KDE packagers: give some love to Cantor

python2_select

I have some posts to write about Cantor but first I would like to request a help to KDE packagers of several Linux distros around the world.

I received some mails from users asking "how can I use python in Cantor?" or "where is python support in Cantor?". Well, python2-backend is available in Cantor since KDE 4.12 release. If you is using KDE >= 4.12 but you can not to use python in Cantor, maybe the package was not build correctly.

python 2 development library (commonly packed as python-devel in some Linux distros) is required to build python2-backend. python 2 is required to use Cantor with python 2.

Then if you are a Cantor user and can not to use Cantor with python, please write a bug report in the bug management system of your distro. You can to put a link in the bug report to this post too.

Anyway, if your distro bring or not bring python2-backend, write a comment below and I will make a table with this information.

24 Apr 2014 12:47am GMT

23 Apr 2014

feedPlanet Python

Stein Magnus Jodal: pyspotify v2.0.0b1 released, with new event loop and audio sinks

pyspotify 2.x is a full rewrite of pyspotify. While pyspotify 1.x is a CPython C extension, pyspotify 2.x uses CFFI to make 100% of the libspotify C library available from Python. It works on CPython 2.7 and 3.2+, as well as PyPy 2.1+.

pyspotify provides a Python interface to Spotify's online music streaming service. With it you can access music metadata, search in Spotify's library of 20+ million tracks, manage your Spotify playlists, and play music from Spotify. All from your own Python applications.

Since the previous release, pyspotify has become thread safe. That is, pyspotify can safely be used from multiple threads. The added thread safety made an integrated event loop possible, which greatly simplifies the usage of pyspotify, as can be seen from the updated example in examples/shell.py. Audio sink helpers for ALSA and PortAudio have been added, together with updated examples that can play music. A number of bugs have been fixed, and at the time of the release, there are no known issues. For more details, see the full changelog.

The pyspotify 2.0.0b1 release marks the completion of all planned features for pyspotify 2.x. The plans for the next releases are fixing bugs as they surface, incrementally improving the documentation, and integrating feedback from increased usage of the library in the wild.

I'd love it if you would give pyspotify 2.x a spin and report any issues you find.

To get started, check out the installation instructions for some help on the C library part of the installation, then continue on with the quickstart guide, and finally the complete API reference.

Happy music hacking!

23 Apr 2014 11:45pm GMT

Stein Magnus Jodal: pyspotify v2.0.0b1 released, with new event loop and audio sinks

pyspotify 2.x is a full rewrite of pyspotify. While pyspotify 1.x is a CPython C extension, pyspotify 2.x uses CFFI to make 100% of the libspotify C library available from Python. It works on CPython 2.7 and 3.2+, as well as PyPy 2.1+.

pyspotify provides a Python interface to Spotify's online music streaming service. With it you can access music metadata, search in Spotify's library of 20+ million tracks, manage your Spotify playlists, and play music from Spotify. All from your own Python applications.

Since the previous release, pyspotify has become thread safe. That is, pyspotify can safely be used from multiple threads. The added thread safety made an integrated event loop possible, which greatly simplifies the usage of pyspotify, as can be seen from the updated example in examples/shell.py. Audio sink helpers for ALSA and PortAudio have been added, together with updated examples that can play music. A number of bugs have been fixed, and at the time of the release, there are no known issues. For more details, see the full changelog.

The pyspotify 2.0.0b1 release marks the completion of all planned features for pyspotify 2.x. The plans for the next releases are fixing bugs as they surface, incrementally improving the documentation, and integrating feedback from increased usage of the library in the wild.

I'd love it if you would give pyspotify 2.x a spin and report any issues you find.

To get started, check out the installation instructions for some help on the C library part of the installation, then continue on with the quickstart guide, and finally the complete API reference.

Happy music hacking!

23 Apr 2014 11:45pm GMT

Shannon -jj Behrens: PyCon Notes: PostgreSQL Proficiency for Python People

In summary, this tutorial was fantastic! I learned more in three hours than I would have learned if I had read a whole book!

Here's the video. Here are the slides. Here are my notes:

Christoph Pettus was the speaker. He's from PostgreSQL Experts.

PostgreSQL is a rich environment.

It's fully ACID compliant.

It has the richest set of features of any modern, production RDMS. It has even more features than
Oracle.

PostgreSQL focuses on quality, security, and spec compliance.

It's capable of very high performance: tens of thousands of transactions per second, petabyte-sized data sets, etc.

To install it, just use your package management system (apt, yum, etc.). Those systems will usually take care of initialization.

There are many options for OS X. Heroku even built a Postgres.app that runs more like a foreground app.

A "cluster" is a single PostgreSQL server (which can manage multiple databases).

initdb creates the basic file structure. PostgreSQL has to be up and running to run initdb.

To create a database:

sudo su - postgres
psql

create database this_new_database;

To drop a database:

drop database this_new_database;

Debian runs initdb for you. Red Hat does not.

Debian has a cluster management system. Use it. See, for instance, pg_createcluster.

Always create databases as UTF-8. Once you've created it, you can't change it.

Don't use SQLASCII. It's a nightmare. Don't use "C locale".

pg_ctl is a built-in command to start and stop PostgreSQL:

cd POSTGRES_DIRECTORY
pg_ctl -D . start

Usually, pg_ctl is wrapped by something provided by your platform.

On Ubuntu, start PostgreSQL via:

service postgresql start

Always use "-m fast" when stopping.

Postgres puts its own data in a top-level directory. Let's call it $PGDATA.

Don't monkey around with that data.

pg_clog and pg_xlog are important. Don't mess with them.

On most systems, configuration lives in $PGDATA.

postgresql.conf contains server configuration.

pg_hba.conf contains authentication settings.

postgresql.conf can feel very overwhelming.

Avoid making a lot of changes to postgresql.conf. Instead, add the following to it:

include "postgresql.conf.include"

Then, mess with "postgresql.conf.include".

The important parameters fall into these categories: logging, memory, checkpoints, and the planner.

Logging:

Be generous with logging. It has a very low impact on the system. It's your best source of info for diagnosing problems.

You can log to syslog or log CSV to files. He showed his typical logging configuration.

He showed his guidelines / heuristics for all the settings, including how to finetune things. They're really good! See his slides.

As of version 9.3, you don't need to tweak Linux kernel parameters anymore.

Do not mess with fsync or synchronous_commit.

Most settings require a server reload to take effect. Some things require a server restart. Some can be set on a per-session basis. Here's how to do that. This is also an example of how to use a transaction:

begin;
set local random_page_cost = 2.5;
show random_page_cost;
abort;

pg_hba.conf contains users and roles. Roles are like groups. They form a hierarchy.

A user is just a role with login privs.

Don't use the "postgres" superuser for anything application-related.

Sadly, you probably will have to grant schema-modification privs to your app user if you use migrations, but if you don't have to, don't.

By default, DB traffic is not encrypted. Turn on SSL if you are running in a cloud provider.

In pg_hba.conf, "trust" means if they can log into the server, they can access Postgres too. "peer" means they can have a Postgres user that matches their username. "md5" is an md5 hash password.

It's a good idea to restrict the IP addresses allowed to talk to the server fairly tightly.

The WAL

The Write-Ahead Log is key to many Postgres operations. It's the basis for replication, crash recovery, etc.

When each transaction is committed, it is logged to the write-ahead log.

Changes in the transaction are flushed to disk.

If the system crashes, the WAL is "replayed" to bring the DB to a consistent state.

It's a continuous record of changes since the last checkpoint.

The WAL is stored in 16MB segments in the pg_xlog directory.

Never delete anything from pg_xlog.

archive_command is a way to move the WAL segments to someplace safe (like a
different system).

By default, synchronous_commit is on, which means that commits do not return until the WAL flush is done. If you turn it off, they'll return when the WAL flush is queued. You might lose transactions in the case of a crash, but there's no risk of database corruption.

Backup and Recovery

Experience has shown that 20% of the time, your EBS volumes will not reattach when you reboot in AWS.

pg_dump is a built-in dump/restore tool.

It takes a logical snapshot of the database.

It doesn't lock the database or prevent writes to disk.

pg_restore restores the database. It's not fast.

It's great for simple backups but not suitable for fast recovery from major failures.

pg_bench is the built in benchmarking tool.

pg_dump -Fc --verbose example > example.dump

Without the -Fc, it dumps SQL commands instead of its custom format.

pg_restore --dbname=example_restored --verbose example.dump

pg_restore takes a long time because it has to recreate indexes.

pg_dumpall --globals-only

Back up each database with pg_dump using --format=custom.

To do a parallel restore, use --jobs=.

If you have a large database, pg_dump may not be appropriate.

A disk snapshot + every WAL segment is enough to recreate the database.

To start a PITR (point in time recovery) backup:

select pg_start_backup(...);

Copy the disk image and any WAL files that are created.

select pg_stop_backup();

Make sure you have all the WAL segments.

The disk image + all the WAL segments are enough to create the DB.

See also github.com/wal-e/wal-e. It's highly recommended.

It automates backups to S3.

He explained how to do a PITR.

With PITR, you can rollback to a particular point in time. You don't have to replay everything.

This is super handy for application failures.

RDS is something that scripts all this stuff for you.

Replication

Send the WAL to another server.

Keep the server up to date with the primary server.

That's how PostgreSQL replication works.

The old way was called "WAL Archiving". Each 16MB segment was sent to the secondary when complete. Use rsync, WAL-E, etc., not scp.

The new way is Streaming Replication.

The secondary gets changes as they happen.

It's all setup via recovery.conf in your $PGDATA.

He showed a recovery.conf for a secondary machine, and showed how to let it become the master.

Always have a disaster recovery strategy.

pg_basebackup is a utility for doing a snapshot of a running server. It's the easiest way to take a snapshot to start a new secondary. It's also useful for archival backups. It's not the fastest thing, but it's pretty foolproof.

Replication:

The good:

Easy to setup.

Schema changes are replicated.

Secondaries can handle read-only queries for load balancing.

It either works or it complains loudly.

The bad:

You get the entire DB cluster or none of it.

No writes of any kind to the secondary, not even temporary tables.

Some things aren't replicated like temporary tables and unlogged tables.

His advice is to start with WAL-E. The README tells you everything. It fixes a ton of problems.

The biggest problem with WAL-E is that writing to S3 can be slow.

Another way to do funky things is trigger-based replication. There's a bunch of third-party packages to do this.

Bucardo is one that lets you do multi-master setups.

However, they're fiddly and complex to set up. They can also fail quietly.

Transactions, MVCC, and Vacuum

BEGIN;
INSERT ...;
INSERT ...;
COMMIT;

By the way, no bank works this way ;)

Everything runs inside of a transaction.

If there is no explicit transaction, each statement is wrapped in one for you.

Everything that modifies the database is transactional, even schema changes.

\d shows you all your tables.

With a transaction, you can even rollback a table drop.

South (the Django migration tool) runs the whole migration in a single transaction.

Many resources are held until the end of a transaction. Keep your transactions brief and to the point.

Beware of "IDLE IN TRANSACTION" sessions. This is a problem for Django apps.

A tuple in Postgres is the same thing as a row.

Postgres uses Multi-Version Concurrency Control. Each transaction sees its own version of the database.

Writers only block writers to the same tuple. Nothing else causes blocking.

Postgres will not allow two snapshots to "fork" the database. If two people try to write to the same tuple, Postgres will block one of them.

There are higher isolation modes. His description of them was really interesting.

He suggested that new apps use SERIALIZABLE. This will help you find the concurrency errors in your app.

Deleted tuples are not usually immediately freed.

Vacuum's primary job is to scavenge tuples that are no longer visible to any transaction.

autovacuum generally handles this problem for you without intervention (since version 8).

Run analyze after a major database change to help the planner out.

If someone tells you "vacuum's not working", they're probably wrong.

The DB generally stabilizes at 20% to 50% bloat. That's acceptable.

The problem might be that there are long-running transactions or idle-in-transaction sessions. They'll block vacuuming. So will manual table locking.

He talked about vacuum issues for rare situations.

Schema Design

Normalization is important, but don't obsess about it.

Pick "entities". Make sure that no entity-level info gets pushed into the subsidiary items.

Pick a naming scheme and stick with it.

Plural or singular? DB people tend to like plural. ORMs tend to like singular.

You probably want lower_case to avoid quoting.

Calculated denormalization can sometimes be useful; copied denormalization is almost never useful.

Joins are good.

PostgreSQL executes joins very efficiently. Don't be afraid of them.

Don't worry about large tables joined with small tables.

Use the typing system. It has a rich set of types.

Use domains to create custom types.

A domain is a core type + a constraint.

Don't use polymorphic fields (fields whose interpretation is dependent on another field).

Don't use strings to store multiple types.

Use constraints. They're cheap and fast.

You can create constraints across multiple columns.

Avoid Entity-Attribute-Value schemas. They cause great pain. They're very inefficient. They make reports very difficult.

Consider using UUIDs instead of serials as synthetic keys.

The problem with serials for keys is that merging tables can be hard.

Don't have "Thing" tables like "Object" tables.

If a table has a few frequently-updated fields and a few slowly-updated fields, consider splitting the table. Split the fast-moving stuff out into a separate 1-to-1 table.

Arrays are a first-class type in PostgreSQL. It's a good substitute for using a subsidiary table.

A list of tags is a good fit for arrays.

He talked about hstore. It's much better than Entity-Attribute-Value. It's great for optional, variable attributes. It's like a hash. It can be indexed, searched, etc. It lets you add attributes to tables for users. Don't use it as a way to avoid all table modifications.

json is now a built in type.

There's also jsonb.

Avoid indexes on big things, like 10k character strings.

NULL it a total pain in the neck.

Only use it to mean "missing value".

Never use it to represent a meaningful value.

Let's call anything 1MB or more a "very large object". Store them in files. Store the metadata in the database. The database API is just not a good fit for this.

Many-to-many tables can get extremely large. Consider replacing them with array fields (either one way or both directions). You can use a trigger to maintain integrity.

You don't want more than about 250k entries in an array.

Use UTF-8. Period.

Always use TIMESTAMPTZ (which Django uses by default). Don't use TIMESTAMP. TIMESTAMPTZ is a timestamp converted to UTC.

Index types:

B-Tree

Use a B-Tree on a column if you frequently query on that column,
use one of the comparison operators, only get back 10-15% of the rows,
and run that query frequently.

It won't use the index if you're going to get back more than 15% of
the rows because it's faster to scan a table then scan an index.

Use a partial index if you can ignore most of the rows.

The entire tuple has to be copied into the index.

GiST

It's a framework to create indexes.

KNN indexes are the K-nearest neighbors.

GIN

Generalized inverted index. Used for full-text search.

The others either are not good or very specific.

Why isn't it using my index?

Use explain analyze to look at the query.

If it thinks it's going to require most of the rows, it'll do a table scan.

If it's wrong, use analyze to update the planner stats.

Sometimes, it can't use the index.

Two ways to create an index:

create index

create index concurrently

reindex rebuilds an index from scratch.

pg_stat_user_indexes tells you about how your indexes are being used.

What do you do if a query is slow:

Use explain or explain analyze.

explain doesn't actually run the query.

"Cost" is measured in arbitrary units. Traditionally, they have been "disk fetches". Costs are inclusive of subnodes.

I think explain analyze actually runs the query.

Things that are bad:

Joins between 2 large tables.

Cross joins (cartesian products). These often happen by accident.

Sequential scans on large tables.

select count(*) is slow because it results in a full table scan since you
have to see if the tuples are alive or dead.

offset / limit. These actually run the query and then throw away that many
rows. Beware that GoogleBot is relentless. Use other keys.

If the database is slow:

Look at pg_stat_activity:

select * from pg_stat_activity;

tail -f the logs.

Too much I/O? iostat 5.

If the database isn't responding:

Try connecting with it using psql.

pg_stat_activity

pg_locks

Python Particulars

psycopg2 is the only real option in Python 2.

The result set of a query is loaded into client memory when the query completes. If there are a ton of rows, you could run out of memory. If you want to scroll through the results, use a "named" cursor. Be sure to dispose of it properly.

The Python 3 situation is not so great. There's py-postgresql. It's pure Python.

If you are using Django 1.6+, use the @atomic decorator.

Cluster all your writes into small transactions. Leave read operations outside.

Do all your writes at the very end of the view function.

Multi-database works very nicely with hot standby.

Point the writes at the primary, and the reads at the secondary.

For Django 1.5, use the @xact decorator.

Sloppy transaction management can cause the dreaded Django idle-in-transaction problem.

Use South for database migration. South is getting merged into Django in version 1.7 of Django.

You can use manual migrations for stuff the Django ORM can't specify.

Special Situations

Upgrade to 9.3.4. Upgrade minor versions promptly.

Major version upgrades require more planning. pg_upgrade has to be run when the database is not running.

A full pg_dump / pg_restore is always the safest, although not the most practical.

Always read the release notes.

All parts of a replication set must be upgraded at once (for major versions).

Use copy, not insert, for bulk loading data. psycopg2 has a nice interface. Do a vacuum afterwards.

AWS

Instances can disappear and come back up without instance storage.

EBS can fail to reattach after reboot.

PIOPS are useful (but pricey) if you are using EBS.

Script everything, instance creation, PostgreSQL, etc. Use Salt. Use a VPC.

Scale up and down as required to meet load. If you're just using them to rent a server, it's really expensive.

PostgreSQL RDS is a managed database instance. Big plus: automatic failover! Big minus: you can't read from the secondary. It's expensive. It's a good place to start.

Sharding

Eventually, you'll run out of write capacity on your master.

postgres-xc is an open source fork of PostgreSQL.

Bucardo provides multi-master write capability.

He talked about custom sharding.

Instagram wrote a nice article about it.

Pooling

Opening a connection is expensive. Use a pooler.

pgbouncer is a pooler.

pgPool II can even do query analysis. However, it has higher overhead and is more complex to configure.

Tools

Monitor everything.

check_postgres.pl is a plugin to monitor PostgreSQL.

pgAdmin III and Navicat are nice clients.

pgbadger is for log analysis. So is pg_stat_statements.

Closing

MVCC works by each tuple having a range of transaction IDs that can see that
tuple.

Failover is annoying to do in the real world. People use HAProxy, some pooler, etc. with some scripting, or they have a human do the failover.

HandyRep is a server-based tool designed to allow you to manage a PostgreSQL "replication cluster", defined as a master and one or more replicas on the same network.

23 Apr 2014 7:41pm GMT

Shannon -jj Behrens: PyCon Notes: PostgreSQL Proficiency for Python People

In summary, this tutorial was fantastic! I learned more in three hours than I would have learned if I had read a whole book!

Here's the video. Here are the slides. Here are my notes:

Christoph Pettus was the speaker. He's from PostgreSQL Experts.

PostgreSQL is a rich environment.

It's fully ACID compliant.

It has the richest set of features of any modern, production RDMS. It has even more features than
Oracle.

PostgreSQL focuses on quality, security, and spec compliance.

It's capable of very high performance: tens of thousands of transactions per second, petabyte-sized data sets, etc.

To install it, just use your package management system (apt, yum, etc.). Those systems will usually take care of initialization.

There are many options for OS X. Heroku even built a Postgres.app that runs more like a foreground app.

A "cluster" is a single PostgreSQL server (which can manage multiple databases).

initdb creates the basic file structure. PostgreSQL has to be up and running to run initdb.

To create a database:

sudo su - postgres
psql

create database this_new_database;

To drop a database:

drop database this_new_database;

Debian runs initdb for you. Red Hat does not.

Debian has a cluster management system. Use it. See, for instance, pg_createcluster.

Always create databases as UTF-8. Once you've created it, you can't change it.

Don't use SQLASCII. It's a nightmare. Don't use "C locale".

pg_ctl is a built-in command to start and stop PostgreSQL:

cd POSTGRES_DIRECTORY
pg_ctl -D . start

Usually, pg_ctl is wrapped by something provided by your platform.

On Ubuntu, start PostgreSQL via:

service postgresql start

Always use "-m fast" when stopping.

Postgres puts its own data in a top-level directory. Let's call it $PGDATA.

Don't monkey around with that data.

pg_clog and pg_xlog are important. Don't mess with them.

On most systems, configuration lives in $PGDATA.

postgresql.conf contains server configuration.

pg_hba.conf contains authentication settings.

postgresql.conf can feel very overwhelming.

Avoid making a lot of changes to postgresql.conf. Instead, add the following to it:

include "postgresql.conf.include"

Then, mess with "postgresql.conf.include".

The important parameters fall into these categories: logging, memory, checkpoints, and the planner.

Logging:

Be generous with logging. It has a very low impact on the system. It's your best source of info for diagnosing problems.

You can log to syslog or log CSV to files. He showed his typical logging configuration.

He showed his guidelines / heuristics for all the settings, including how to finetune things. They're really good! See his slides.

As of version 9.3, you don't need to tweak Linux kernel parameters anymore.

Do not mess with fsync or synchronous_commit.

Most settings require a server reload to take effect. Some things require a server restart. Some can be set on a per-session basis. Here's how to do that. This is also an example of how to use a transaction:

begin;
set local random_page_cost = 2.5;
show random_page_cost;
abort;

pg_hba.conf contains users and roles. Roles are like groups. They form a hierarchy.

A user is just a role with login privs.

Don't use the "postgres" superuser for anything application-related.

Sadly, you probably will have to grant schema-modification privs to your app user if you use migrations, but if you don't have to, don't.

By default, DB traffic is not encrypted. Turn on SSL if you are running in a cloud provider.

In pg_hba.conf, "trust" means if they can log into the server, they can access Postgres too. "peer" means they can have a Postgres user that matches their username. "md5" is an md5 hash password.

It's a good idea to restrict the IP addresses allowed to talk to the server fairly tightly.

The WAL

The Write-Ahead Log is key to many Postgres operations. It's the basis for replication, crash recovery, etc.

When each transaction is committed, it is logged to the write-ahead log.

Changes in the transaction are flushed to disk.

If the system crashes, the WAL is "replayed" to bring the DB to a consistent state.

It's a continuous record of changes since the last checkpoint.

The WAL is stored in 16MB segments in the pg_xlog directory.

Never delete anything from pg_xlog.

archive_command is a way to move the WAL segments to someplace safe (like a
different system).

By default, synchronous_commit is on, which means that commits do not return until the WAL flush is done. If you turn it off, they'll return when the WAL flush is queued. You might lose transactions in the case of a crash, but there's no risk of database corruption.

Backup and Recovery

Experience has shown that 20% of the time, your EBS volumes will not reattach when you reboot in AWS.

pg_dump is a built-in dump/restore tool.

It takes a logical snapshot of the database.

It doesn't lock the database or prevent writes to disk.

pg_restore restores the database. It's not fast.

It's great for simple backups but not suitable for fast recovery from major failures.

pg_bench is the built in benchmarking tool.

pg_dump -Fc --verbose example > example.dump

Without the -Fc, it dumps SQL commands instead of its custom format.

pg_restore --dbname=example_restored --verbose example.dump

pg_restore takes a long time because it has to recreate indexes.

pg_dumpall --globals-only

Back up each database with pg_dump using --format=custom.

To do a parallel restore, use --jobs=.

If you have a large database, pg_dump may not be appropriate.

A disk snapshot + every WAL segment is enough to recreate the database.

To start a PITR (point in time recovery) backup:

select pg_start_backup(...);

Copy the disk image and any WAL files that are created.

select pg_stop_backup();

Make sure you have all the WAL segments.

The disk image + all the WAL segments are enough to create the DB.

See also github.com/wal-e/wal-e. It's highly recommended.

It automates backups to S3.

He explained how to do a PITR.

With PITR, you can rollback to a particular point in time. You don't have to replay everything.

This is super handy for application failures.

RDS is something that scripts all this stuff for you.

Replication

Send the WAL to another server.

Keep the server up to date with the primary server.

That's how PostgreSQL replication works.

The old way was called "WAL Archiving". Each 16MB segment was sent to the secondary when complete. Use rsync, WAL-E, etc., not scp.

The new way is Streaming Replication.

The secondary gets changes as they happen.

It's all setup via recovery.conf in your $PGDATA.

He showed a recovery.conf for a secondary machine, and showed how to let it become the master.

Always have a disaster recovery strategy.

pg_basebackup is a utility for doing a snapshot of a running server. It's the easiest way to take a snapshot to start a new secondary. It's also useful for archival backups. It's not the fastest thing, but it's pretty foolproof.

Replication:

The good:

Easy to setup.

Schema changes are replicated.

Secondaries can handle read-only queries for load balancing.

It either works or it complains loudly.

The bad:

You get the entire DB cluster or none of it.

No writes of any kind to the secondary, not even temporary tables.

Some things aren't replicated like temporary tables and unlogged tables.

His advice is to start with WAL-E. The README tells you everything. It fixes a ton of problems.

The biggest problem with WAL-E is that writing to S3 can be slow.

Another way to do funky things is trigger-based replication. There's a bunch of third-party packages to do this.

Bucardo is one that lets you do multi-master setups.

However, they're fiddly and complex to set up. They can also fail quietly.

Transactions, MVCC, and Vacuum

BEGIN;
INSERT ...;
INSERT ...;
COMMIT;

By the way, no bank works this way ;)

Everything runs inside of a transaction.

If there is no explicit transaction, each statement is wrapped in one for you.

Everything that modifies the database is transactional, even schema changes.

\d shows you all your tables.

With a transaction, you can even rollback a table drop.

South (the Django migration tool) runs the whole migration in a single transaction.

Many resources are held until the end of a transaction. Keep your transactions brief and to the point.

Beware of "IDLE IN TRANSACTION" sessions. This is a problem for Django apps.

A tuple in Postgres is the same thing as a row.

Postgres uses Multi-Version Concurrency Control. Each transaction sees its own version of the database.

Writers only block writers to the same tuple. Nothing else causes blocking.

Postgres will not allow two snapshots to "fork" the database. If two people try to write to the same tuple, Postgres will block one of them.

There are higher isolation modes. His description of them was really interesting.

He suggested that new apps use SERIALIZABLE. This will help you find the concurrency errors in your app.

Deleted tuples are not usually immediately freed.

Vacuum's primary job is to scavenge tuples that are no longer visible to any transaction.

autovacuum generally handles this problem for you without intervention (since version 8).

Run analyze after a major database change to help the planner out.

If someone tells you "vacuum's not working", they're probably wrong.

The DB generally stabilizes at 20% to 50% bloat. That's acceptable.

The problem might be that there are long-running transactions or idle-in-transaction sessions. They'll block vacuuming. So will manual table locking.

He talked about vacuum issues for rare situations.

Schema Design

Normalization is important, but don't obsess about it.

Pick "entities". Make sure that no entity-level info gets pushed into the subsidiary items.

Pick a naming scheme and stick with it.

Plural or singular? DB people tend to like plural. ORMs tend to like singular.

You probably want lower_case to avoid quoting.

Calculated denormalization can sometimes be useful; copied denormalization is almost never useful.

Joins are good.

PostgreSQL executes joins very efficiently. Don't be afraid of them.

Don't worry about large tables joined with small tables.

Use the typing system. It has a rich set of types.

Use domains to create custom types.

A domain is a core type + a constraint.

Don't use polymorphic fields (fields whose interpretation is dependent on another field).

Don't use strings to store multiple types.

Use constraints. They're cheap and fast.

You can create constraints across multiple columns.

Avoid Entity-Attribute-Value schemas. They cause great pain. They're very inefficient. They make reports very difficult.

Consider using UUIDs instead of serials as synthetic keys.

The problem with serials for keys is that merging tables can be hard.

Don't have "Thing" tables like "Object" tables.

If a table has a few frequently-updated fields and a few slowly-updated fields, consider splitting the table. Split the fast-moving stuff out into a separate 1-to-1 table.

Arrays are a first-class type in PostgreSQL. It's a good substitute for using a subsidiary table.

A list of tags is a good fit for arrays.

He talked about hstore. It's much better than Entity-Attribute-Value. It's great for optional, variable attributes. It's like a hash. It can be indexed, searched, etc. It lets you add attributes to tables for users. Don't use it as a way to avoid all table modifications.

json is now a built in type.

There's also jsonb.

Avoid indexes on big things, like 10k character strings.

NULL it a total pain in the neck.

Only use it to mean "missing value".

Never use it to represent a meaningful value.

Let's call anything 1MB or more a "very large object". Store them in files. Store the metadata in the database. The database API is just not a good fit for this.

Many-to-many tables can get extremely large. Consider replacing them with array fields (either one way or both directions). You can use a trigger to maintain integrity.

You don't want more than about 250k entries in an array.

Use UTF-8. Period.

Always use TIMESTAMPTZ (which Django uses by default). Don't use TIMESTAMP. TIMESTAMPTZ is a timestamp converted to UTC.

Index types:

B-Tree

Use a B-Tree on a column if you frequently query on that column,
use one of the comparison operators, only get back 10-15% of the rows,
and run that query frequently.

It won't use the index if you're going to get back more than 15% of
the rows because it's faster to scan a table then scan an index.

Use a partial index if you can ignore most of the rows.

The entire tuple has to be copied into the index.

GiST

It's a framework to create indexes.

KNN indexes are the K-nearest neighbors.

GIN

Generalized inverted index. Used for full-text search.

The others either are not good or very specific.

Why isn't it using my index?

Use explain analyze to look at the query.

If it thinks it's going to require most of the rows, it'll do a table scan.

If it's wrong, use analyze to update the planner stats.

Sometimes, it can't use the index.

Two ways to create an index:

create index

create index concurrently

reindex rebuilds an index from scratch.

pg_stat_user_indexes tells you about how your indexes are being used.

What do you do if a query is slow:

Use explain or explain analyze.

explain doesn't actually run the query.

"Cost" is measured in arbitrary units. Traditionally, they have been "disk fetches". Costs are inclusive of subnodes.

I think explain analyze actually runs the query.

Things that are bad:

Joins between 2 large tables.

Cross joins (cartesian products). These often happen by accident.

Sequential scans on large tables.

select count(*) is slow because it results in a full table scan since you
have to see if the tuples are alive or dead.

offset / limit. These actually run the query and then throw away that many
rows. Beware that GoogleBot is relentless. Use other keys.

If the database is slow:

Look at pg_stat_activity:

select * from pg_stat_activity;

tail -f the logs.

Too much I/O? iostat 5.

If the database isn't responding:

Try connecting with it using psql.

pg_stat_activity

pg_locks

Python Particulars

psycopg2 is the only real option in Python 2.

The result set of a query is loaded into client memory when the query completes. If there are a ton of rows, you could run out of memory. If you want to scroll through the results, use a "named" cursor. Be sure to dispose of it properly.

The Python 3 situation is not so great. There's py-postgresql. It's pure Python.

If you are using Django 1.6+, use the @atomic decorator.

Cluster all your writes into small transactions. Leave read operations outside.

Do all your writes at the very end of the view function.

Multi-database works very nicely with hot standby.

Point the writes at the primary, and the reads at the secondary.

For Django 1.5, use the @xact decorator.

Sloppy transaction management can cause the dreaded Django idle-in-transaction problem.

Use South for database migration. South is getting merged into Django in version 1.7 of Django.

You can use manual migrations for stuff the Django ORM can't specify.

Special Situations

Upgrade to 9.3.4. Upgrade minor versions promptly.

Major version upgrades require more planning. pg_upgrade has to be run when the database is not running.

A full pg_dump / pg_restore is always the safest, although not the most practical.

Always read the release notes.

All parts of a replication set must be upgraded at once (for major versions).

Use copy, not insert, for bulk loading data. psycopg2 has a nice interface. Do a vacuum afterwards.

AWS

Instances can disappear and come back up without instance storage.

EBS can fail to reattach after reboot.

PIOPS are useful (but pricey) if you are using EBS.

Script everything, instance creation, PostgreSQL, etc. Use Salt. Use a VPC.

Scale up and down as required to meet load. If you're just using them to rent a server, it's really expensive.

PostgreSQL RDS is a managed database instance. Big plus: automatic failover! Big minus: you can't read from the secondary. It's expensive. It's a good place to start.

Sharding

Eventually, you'll run out of write capacity on your master.

postgres-xc is an open source fork of PostgreSQL.

Bucardo provides multi-master write capability.

He talked about custom sharding.

Instagram wrote a nice article about it.

Pooling

Opening a connection is expensive. Use a pooler.

pgbouncer is a pooler.

pgPool II can even do query analysis. However, it has higher overhead and is more complex to configure.

Tools

Monitor everything.

check_postgres.pl is a plugin to monitor PostgreSQL.

pgAdmin III and Navicat are nice clients.

pgbadger is for log analysis. So is pg_stat_statements.

Closing

MVCC works by each tuple having a range of transaction IDs that can see that
tuple.

Failover is annoying to do in the real world. People use HAProxy, some pooler, etc. with some scripting, or they have a human do the failover.

HandyRep is a server-based tool designed to allow you to manage a PostgreSQL "replication cluster", defined as a master and one or more replicas on the same network.

23 Apr 2014 7:41pm GMT

End Point: Dictionary Comprehensions in Python

Python has many features which usually stay unknown to many programmers. This time I discovered the dictionary comprehensions. One of the nice features which can make code much cleaner. Let me start with a simple introduction about list comprehensions and collection uniqueness

List Comprehensions

List comprehensions are a much simpler way of creating lists. This is one feature, rather widely used, and I saw this in many examples and source of many libraries.

Imagine you have a function which returns a list of data. A good example of this is xrange(start, end) function, which returns all numbers within the range [start, end). This is a generator, so it doesn't return all numbers at once, but you need to call this function many times, and each time it returns the next number.

Getting all numbers from the range [1, 10] using this function can be done like this:

numbers = []
for i in xrange(1, 11):
    numbers.append(i)

If you want to get only the even numbers, then you can write:

numbers = []
for i in xrange(1, 11):
    if i % 2 == 0:
        numbers.append(i)

List comprehensions can make the code much simpler. The below expression evaluates to a list:

[ expression for item in list if conditional ]

The first example can be then written as:

numbers = [i for i in xrange(1, 11)]

and the second:

numbers = [i for i in xrange(1, 11) if i % 2 == 0]

Of course this syntax can be a little bit strange at the very first moment, but you can get used to it, and then the code can be much simpler.

Removing Duplicates

Another common usage of collections is to remove duplicates. And again there are plenty of ways to do it.

Consider a collection like this:

numbers = [i for i in xrange(1,11)] + [i for i in xrange(1,6)]

The most complicated way of removing duplicates I've ever seen was:

unique_numbers = []
for n in numbers:
    if n not in unique_numbers:
        unique_numbers.append(n)

Of course it works, but there a much easier way. You can use a standard type like set. Set cannot have duplicates, so when converting a list to a set, all duplicates are removed. However at the end there will be set, not list, if you want to have a list, then you should convert it again:

unique_numbers = list(set(numbers))

Removing Object Duplicates

With objects, or dictionaries, the situation is a little bit different. You can have a list of dictionaries, where you use just one field for identity, this can look like:

data = [
  {'id': 10, 'data': '...'},
  {'id': 11, 'data': '...'},
  {'id': 12, 'data': '...'},
  {'id': 10, 'data': '...'},
  {'id': 11, 'data': '...'},
]

Removing duplicates, again, can be done using more or less code. Less is better, of course. With more code it can be:

unique_data = []
for d in data:
    data_exists = False
    for ud in unique_data:
        if ud['id'] == d['id']:
          data_exists = True
          break
    if not data_exists:
        unique_data.append(d)

And this can be done using a thing I discoverd a couple of days ago, this is the dictionary comprehension. It has a similar syntax to the list comprehension, however it evaluates to a dicionary:

{ key:value for item in list if conditional }

This can be used to make a list without all duplicates using a custom field a identity:

{ d['id']:d for d in data }.values()

The above code creates a dictionary with a key, which is the field I want to use for uniqueness, and the whole dictionary as the value. The dictionary then contains only one entry for each key. The values() function is used to get a list only with values, as I don't need the key:value mappings any more.

23 Apr 2014 1:37pm GMT

End Point: Dictionary Comprehensions in Python

Python has many features which usually stay unknown to many programmers. This time I discovered the dictionary comprehensions. One of the nice features which can make code much cleaner. Let me start with a simple introduction about list comprehensions and collection uniqueness

List Comprehensions

List comprehensions are a much simpler way of creating lists. This is one feature, rather widely used, and I saw this in many examples and source of many libraries.

Imagine you have a function which returns a list of data. A good example of this is xrange(start, end) function, which returns all numbers within the range [start, end). This is a generator, so it doesn't return all numbers at once, but you need to call this function many times, and each time it returns the next number.

Getting all numbers from the range [1, 10] using this function can be done like this:

numbers = []
for i in xrange(1, 11):
    numbers.append(i)

If you want to get only the even numbers, then you can write:

numbers = []
for i in xrange(1, 11):
    if i % 2 == 0:
        numbers.append(i)

List comprehensions can make the code much simpler. The below expression evaluates to a list:

[ expression for item in list if conditional ]

The first example can be then written as:

numbers = [i for i in xrange(1, 11)]

and the second:

numbers = [i for i in xrange(1, 11) if i % 2 == 0]

Of course this syntax can be a little bit strange at the very first moment, but you can get used to it, and then the code can be much simpler.

Removing Duplicates

Another common usage of collections is to remove duplicates. And again there are plenty of ways to do it.

Consider a collection like this:

numbers = [i for i in xrange(1,11)] + [i for i in xrange(1,6)]

The most complicated way of removing duplicates I've ever seen was:

unique_numbers = []
for n in numbers:
    if n not in unique_numbers:
        unique_numbers.append(n)

Of course it works, but there a much easier way. You can use a standard type like set. Set cannot have duplicates, so when converting a list to a set, all duplicates are removed. However at the end there will be set, not list, if you want to have a list, then you should convert it again:

unique_numbers = list(set(numbers))

Removing Object Duplicates

With objects, or dictionaries, the situation is a little bit different. You can have a list of dictionaries, where you use just one field for identity, this can look like:

data = [
  {'id': 10, 'data': '...'},
  {'id': 11, 'data': '...'},
  {'id': 12, 'data': '...'},
  {'id': 10, 'data': '...'},
  {'id': 11, 'data': '...'},
]

Removing duplicates, again, can be done using more or less code. Less is better, of course. With more code it can be:

unique_data = []
for d in data:
    data_exists = False
    for ud in unique_data:
        if ud['id'] == d['id']:
          data_exists = True
          break
    if not data_exists:
        unique_data.append(d)

And this can be done using a thing I discoverd a couple of days ago, this is the dictionary comprehension. It has a similar syntax to the list comprehension, however it evaluates to a dicionary:

{ key:value for item in list if conditional }

This can be used to make a list without all duplicates using a custom field a identity:

{ d['id']:d for d in data }.values()

The above code creates a dictionary with a key, which is the field I want to use for uniqueness, and the whole dictionary as the value. The dictionary then contains only one entry for each key. The values() function is used to get a list only with values, as I don't need the key:value mappings any more.

23 Apr 2014 1:37pm GMT

Gael Varoquaux: Google summer of code projects for scikit-learn

I'd like to welcome the four students that were accepted for the GSoC this year:

Welcome to all of you. Your submissions were excellent, and you demonstrated a good will to integrate in the project, with its social and coding dynamics. It is a privilege to work with you.

I'd also like to thank all the mentors, Alex, Arnaud, Daniel, James, Jaidev, Olivier, Robert and Vlad. It is a lot of work to mentor and mentors are not only making it possible for great code to enter scikit-learn, but also shaping a future generation of scikit-learn contributors.

23 Apr 2014 5:56am GMT

Gael Varoquaux: Google summer of code projects for scikit-learn

I'd like to welcome the four students that were accepted for the GSoC this year:

Welcome to all of you. Your submissions were excellent, and you demonstrated a good will to integrate in the project, with its social and coding dynamics. It is a privilege to work with you.

I'd also like to thank all the mentors, Alex, Arnaud, Daniel, James, Jaidev, Olivier, Robert and Vlad. It is a lot of work to mentor and mentors are not only making it possible for great code to enter scikit-learn, but also shaping a future generation of scikit-learn contributors.

23 Apr 2014 5:56am GMT

Europython: Financial assistance, partner programme, diversity and schedule

Financial Assistance Programme

In the second round of the EuroPython Financial Assistance Programme we granted an additional sum of 3000 EUR for April which results in a total sum of 8000 EUR so far. There will be a third last round in May. So if you are seeking for financial support for coming to the EuroPython 2014 in Berlin you can still apply for the last round.

We also started the sale of Financial Assistance support tickets last month. You can buy one or more Financial Assistance support tickets for a value of 10, 20, 50 or 100 Euro. With the purchase of Financial Assistance tickets you will support further Python enthusiasts to go to EuroPython 2014 in case they need financial support for a ticket or travel expenses. The EuroPython organization team granted already 5.000 Euro last month in the first of three rounds. Please show your appreciation for the Python community and help us to bring more people to the conference which could not make it otherwise.

Partner programme

We are currently working on the details for a partner & family programme in case you want to bring your better half or your whole family to Berlin. Berlin offers a lot of options for spending the time outside the conference programme like visits to museums, boat trips or guided trips to the Berliner "Unterwelten" and many other places of interest. A survey will be announced and published shortly in order to involve you and your partner into the arrangement and selection of the partner programme.

Diversity at EuroPython

There has been a heated discussion over the last week about the representation of women as speakers at EuroPython 2014 and about diversity in general. You can find the official thread here and the official statement of the EuroPython organizers here. Starting point of the discussion was that PyCon 2014 in Montreal reached an outstanding attendance of about 30% women (both as speaker and attendees).

The EuroPython organizers are committed to diversity by gender, sexual orientation and location. Besides the existing Code of Conduct we are working on an explicit Diversity Statement in order to make the EuroPython 2014 enjoyable for everyone. Making EuroPython more diverse is a long-time process that involves the EuroPython Society, the local organizers and the European Python community as a whole. Diversity and outreach can not be dictated but must be lived in reality on all levels.

Schedule

Therehas been some confusion related to the published preliminary schedule (also under the aspects of diversity). The complete schedule will be announced shortly with roughly additional 50 talks and all trainings. The programme team is currently heavily working on the final version that will also take more diversity aspects into account for choosing the talks. We apologize for the confusion - specially to all persons that submitted a proposal keeping them in an uninformed state. The problem was caused by some internal miscommunication but also by the desire to publish a resilient schedule as early as possible in order to let the speakers make their travel arrangements in time.

23 Apr 2014 5:39am GMT

Europython: Financial assistance, partner programme, diversity and schedule

Financial Assistance Programme

In the second round of the EuroPython Financial Assistance Programme we granted an additional sum of 3000 EUR for April which results in a total sum of 8000 EUR so far. There will be a third last round in May. So if you are seeking for financial support for coming to the EuroPython 2014 in Berlin you can still apply for the last round.

We also started the sale of Financial Assistance support tickets last month. You can buy one or more Financial Assistance support tickets for a value of 10, 20, 50 or 100 Euro. With the purchase of Financial Assistance tickets you will support further Python enthusiasts to go to EuroPython 2014 in case they need financial support for a ticket or travel expenses. The EuroPython organization team granted already 5.000 Euro last month in the first of three rounds. Please show your appreciation for the Python community and help us to bring more people to the conference which could not make it otherwise.

Partner programme

We are currently working on the details for a partner & family programme in case you want to bring your better half or your whole family to Berlin. Berlin offers a lot of options for spending the time outside the conference programme like visits to museums, boat trips or guided trips to the Berliner "Unterwelten" and many other places of interest. A survey will be announced and published shortly in order to involve you and your partner into the arrangement and selection of the partner programme.

Diversity at EuroPython

There has been a heated discussion over the last week about the representation of women as speakers at EuroPython 2014 and about diversity in general. You can find the official thread here and the official statement of the EuroPython organizers here. Starting point of the discussion was that PyCon 2014 in Montreal reached an outstanding attendance of about 30% women (both as speaker and attendees).

The EuroPython organizers are committed to diversity by gender, sexual orientation and location. Besides the existing Code of Conduct we are working on an explicit Diversity Statement in order to make the EuroPython 2014 enjoyable for everyone. Making EuroPython more diverse is a long-time process that involves the EuroPython Society, the local organizers and the European Python community as a whole. Diversity and outreach can not be dictated but must be lived in reality on all levels.

Schedule

Therehas been some confusion related to the published preliminary schedule (also under the aspects of diversity). The complete schedule will be announced shortly with roughly additional 50 talks and all trainings. The programme team is currently heavily working on the final version that will also take more diversity aspects into account for choosing the talks. We apologize for the confusion - specially to all persons that submitted a proposal keeping them in an uninformed state. The problem was caused by some internal miscommunication but also by the desire to publish a resilient schedule as early as possible in order to let the speakers make their travel arrangements in time.

23 Apr 2014 5:39am GMT

11 Oct 2013

feedPython Software Foundation | GSoC'11 Students

Yeswanth Swami: How I kicked off GSoC

Zero to hero

What Prompted me??

I started my third year thinking I should do something that would put me different from the rest and one of my professors suggested me as to why don't I apply for GSoC. I don't know why but I took the suggestion rather seriously, thanks to the bet I had with one of my friend(who is about to complete his MBBS) that whoever earns first will buy the other a "RayBan shades". Well, that's it. I was determined. I started my research early, probably during the start of February(I knew I want to buy my friend, his shades and also buy mine too, in the process).

What experiences I had before??

I started looking at previous years' GSoC projects(having had little experience with Open Source) and started learning how to contribute. I was also very fascinated to the amount of knowledge one could gain just by googling and browsing web pages . I discovered very soon, as to what an immensely great tool , email, through which I could chat with anyone in the open source world and ask seemingly stupid questions and always expect to get a gentle reply back with an answer. Well, that held me spell bound and I knew I want to contribute to Open Source.

How did I begin??

About the middle of March, I discovered that my passion for Python as a programming language increased , after understanding how easy it is as a language. Added to that, my popularity among my fellow classmates increased when I started evangelizing Python(thanks to my seniors for introducing it, I guess I did a decent job popularizing the language). And I started contributing to PSF(Python Software Foundation) , started with a simple bug to fix documentation and slowly my interactivity in IRC increased and I started liking one of the project one of the community member proposed.

A twist in the story??

There I was, still a noob and not knowing how to convince my probable mentor that I could complete the project, given direction. About this juncture, a fellow student(from some university in France) mailed this particular mentor that he was interested in the project . Do, remember, I was part of the mailing list and follow the happenings of it. So, I was furious knowing that I had a competition(having put so much effort) and I was not willing to compromise my project (knowing that this is the one project I actually understood and started researching a little bit too). The other projects require me to have some domain knowledge. I went back to my teachers, seniors, friends and Google and started asking the question , "how would i solve the problem the mentor posted?" . I framed a couple of answers, though very noobish , but at least I could reply the email thread posting my understanding of the problem and how I would solve it and also ask various questions I had in my mind. Well, the mentor replied, immediately to my surprise, and responded back with comments as well as answers to the questions I posed. Again, my nemesis/competitor replied back(he having good knowledge about the problem domain). I knew it was not going to be easy. Hence, I went back again, through all my sources, made further understanding of the problem and posted back again. I guess, about 20 mails in the thread , till we(all three of us) decided we should catch up in IRC and discuss more.

The conclusion:

Well, at IRC , most of senior members from the community were present, and they suggested that they should probably extend the scope of the project(since two students were interested in one project and showed immense passion). Unsurprisingly, over multiple meetings, the project scope was expanded, both the students were given equal important but independent tasks and both the students got opportunity to say they are Google Summer of Code students. Thank goodness, we decided to built the project from scratch giving us more than enough work on our plate.

Movie titles:

1) In the open source world, there is no competition , it is only "COLLABORATION".

2) Why give up, when you can win??

3) From Zero to Hero!!

4) A prodigy in making

p.s. I still owe my friend his shades . *sshole, I am still waiting for him to visit me so that I can buy him his shades and buy mine too. Also, I know its been two years since the story happened, but it is never too late to share, don't you agree??


11 Oct 2013 5:39am GMT