22 Sep 2017

feedPlanet Python

Django Weblog: Django 2.0 alpha 1 released

Django 2.0 alpha 1 is now available. It represents the first stage in the 2.0 release cycle and is an opportunity for you to try out the changes coming in Django 2.0.

Django 2.0 has an assortment of new features which you can read about in the in-development 2.0 release notes.

This alpha milestone marks a complete feature freeze. The current release schedule calls for a beta release in about a month and a release candidate about a month from then. We'll only be able to keep this schedule if we get early and often testing from the community. Updates on the release schedule schedule are available on the django-developers mailing list.

As with all alpha and beta packages, this is not for production use. But if you'd like to take some of the new features for a spin, or to help find and fix bugs (which should be reported to the issue tracker), you can grab a copy of the alpha package from our downloads page or on PyPI.

The PGP key ID used for this release is Tim Graham: 1E8ABDC773EDE252.

22 Sep 2017 6:32pm GMT

Django Weblog: Django 2.0 alpha 1 released

Django 2.0 alpha 1 is now available. It represents the first stage in the 2.0 release cycle and is an opportunity for you to try out the changes coming in Django 2.0.

Django 2.0 has an assortment of new features which you can read about in the in-development 2.0 release notes.

This alpha milestone marks a complete feature freeze. The current release schedule calls for a beta release in about a month and a release candidate about a month from then. We'll only be able to keep this schedule if we get early and often testing from the community. Updates on the release schedule schedule are available on the django-developers mailing list.

As with all alpha and beta packages, this is not for production use. But if you'd like to take some of the new features for a spin, or to help find and fix bugs (which should be reported to the issue tracker), you can grab a copy of the alpha package from our downloads page or on PyPI.

The PGP key ID used for this release is Tim Graham: 1E8ABDC773EDE252.

22 Sep 2017 6:32pm GMT

PyCharm: Interview: Kenneth Love on testing, Python, and organizing

Python is one of the easiest languages to learn, but that doesn't mean all the programming concepts become easy. One that's hard: testing.

Kenneth Love, well-known speaker, organizer, and teacher in Python, has years of experience bringing complicated subjects within reach. He'll be our guest for a webinar on Visual Testing With PyCharm. He'll introduce testing concepts and show how PyCharm makes it easier by putting a visual face on testing.

We interviewed Kenneth to help set the scene for the webinar.

It's a pleasure having you with us for a webinar. My son and I spent summer vacation going through your Python Basics course twice and it was a big help. Do you get that a lot?

Yeah, I've had quite a few people thank me for making that course, and others that I've made, over the years. It seems to be a course that has really caught people that weren't able to get going at Python through other means.

Give us a little background on yourself, your Python origin story and what you're doing now.

I've been using Python for 8-10 years now. I picked it up, almost on a whim, after a friend and I both kind of soured on Ruby. I made the mistake of learning Django before I really learned Python. Eventually, though, I figured out that I should know how more than just an ORM and HTTP works and learned the core parts of the language. I've been a full-time Python developer or teacher for the past six or seven years.

I just recently changed jobs and left teaching to go back into the world of development. Things are changing all the time and I felt like I needed to get back to development for awhile.

Is testing hard to teach?

I think it's a tricky area to teach. People are used to writing and running code. But writing code that runs your code and looks at what happens? It's just a bit too meta for a lot of beginners. It's also the case that a lot of beginners write code that's hard to test, through no fault of their own (they're beginners! I know I wrote messy code when I was first starting) so that can really increase the complexity of learning to test.

Do you have special tricks that seem to work on conceptually hard topics? Debugging, testing, things like that.

Like with most topics, you have to strip away the magic and the special words. We often have a problem in tech of gatekeeping with jargon. We use words that have very specific meanings, like say "fixtures", and we expect everyone to just know them. When someone is new to testing, the term "fixtures" might not have any meaning at all, or mean something completely unrelated. If you can define these terms for them, make them familiar and comfortable, then they can get onto the business of learning this new technique a lot faster.

You're also well-known for organizing. Can you give us your origin story on how you got involved?

I honestly just kind of stumbled into it! My friend Lacey Williams Henschel wanted to do a Django Girls workshop here in Portland. The company I worked for had a great space for it, so I volunteered it. She wanted a co-organizer so I volunteered for that, too. We had a great event and went on to do two more of them after that.

I got into organizing DjangoCon US by being a pain in the butt. I was a pretty big critic of the previous incarnation of DjangoCon US, before it became community-ran. So when the community took it over, I kinda got tapped to help out (no one forced me, of course, but I felt like I had to step up). Since then I've been involved in the program selection for the past two years and did a lot of speaker encouragement and talk rating before that.

Django Girls is such a wonderful story. Mind giving us some background and status on that?

Honestly, I'm not heavily involved with Django Girls any longer. Lacey and I organized the three events and then we both mostly bowed out. It's definitely an amazing organization and I'm completely astounded by the numbers they recently posted. I would love to see more DG-like organizations pop up for other languages and frameworks.

Let's close out with kind of an open-ended question. What's something you think you'll study a lot more in the next year?

I'm really interested in this recent upswing of languages that are focused more on speed and safety. I'd really like to spend some time digging into the Rust programming language. Due to work, I'll likely be learning more about React and Go as well.

22 Sep 2017 3:00pm GMT

PyCharm: Interview: Kenneth Love on testing, Python, and organizing

Python is one of the easiest languages to learn, but that doesn't mean all the programming concepts become easy. One that's hard: testing.

Kenneth Love, well-known speaker, organizer, and teacher in Python, has years of experience bringing complicated subjects within reach. He'll be our guest for a webinar on Visual Testing With PyCharm. He'll introduce testing concepts and show how PyCharm makes it easier by putting a visual face on testing.

We interviewed Kenneth to help set the scene for the webinar.

It's a pleasure having you with us for a webinar. My son and I spent summer vacation going through your Python Basics course twice and it was a big help. Do you get that a lot?

Yeah, I've had quite a few people thank me for making that course, and others that I've made, over the years. It seems to be a course that has really caught people that weren't able to get going at Python through other means.

Give us a little background on yourself, your Python origin story and what you're doing now.

I've been using Python for 8-10 years now. I picked it up, almost on a whim, after a friend and I both kind of soured on Ruby. I made the mistake of learning Django before I really learned Python. Eventually, though, I figured out that I should know how more than just an ORM and HTTP works and learned the core parts of the language. I've been a full-time Python developer or teacher for the past six or seven years.

I just recently changed jobs and left teaching to go back into the world of development. Things are changing all the time and I felt like I needed to get back to development for awhile.

Is testing hard to teach?

I think it's a tricky area to teach. People are used to writing and running code. But writing code that runs your code and looks at what happens? It's just a bit too meta for a lot of beginners. It's also the case that a lot of beginners write code that's hard to test, through no fault of their own (they're beginners! I know I wrote messy code when I was first starting) so that can really increase the complexity of learning to test.

Do you have special tricks that seem to work on conceptually hard topics? Debugging, testing, things like that.

Like with most topics, you have to strip away the magic and the special words. We often have a problem in tech of gatekeeping with jargon. We use words that have very specific meanings, like say "fixtures", and we expect everyone to just know them. When someone is new to testing, the term "fixtures" might not have any meaning at all, or mean something completely unrelated. If you can define these terms for them, make them familiar and comfortable, then they can get onto the business of learning this new technique a lot faster.

You're also well-known for organizing. Can you give us your origin story on how you got involved?

I honestly just kind of stumbled into it! My friend Lacey Williams Henschel wanted to do a Django Girls workshop here in Portland. The company I worked for had a great space for it, so I volunteered it. She wanted a co-organizer so I volunteered for that, too. We had a great event and went on to do two more of them after that.

I got into organizing DjangoCon US by being a pain in the butt. I was a pretty big critic of the previous incarnation of DjangoCon US, before it became community-ran. So when the community took it over, I kinda got tapped to help out (no one forced me, of course, but I felt like I had to step up). Since then I've been involved in the program selection for the past two years and did a lot of speaker encouragement and talk rating before that.

Django Girls is such a wonderful story. Mind giving us some background and status on that?

Honestly, I'm not heavily involved with Django Girls any longer. Lacey and I organized the three events and then we both mostly bowed out. It's definitely an amazing organization and I'm completely astounded by the numbers they recently posted. I would love to see more DG-like organizations pop up for other languages and frameworks.

Let's close out with kind of an open-ended question. What's something you think you'll study a lot more in the next year?

I'm really interested in this recent upswing of languages that are focused more on speed and safety. I'd really like to spend some time digging into the Rust programming language. Due to work, I'll likely be learning more about React and Go as well.

22 Sep 2017 3:00pm GMT

NumFOCUS: Open Journals joins NumFOCUS Sponsored Projects

NumFOCUS is pleased to welcome the Open Journals as a fiscally sponsored project. Open Journals is a collection of open source, open access journals. The team behind Open Journals believes that code review and high-quality, reusable software are a critical-but often overlooked-part of academic research. The primary goal of Open Journals is to provide venues […]

22 Sep 2017 2:00pm GMT

NumFOCUS: Open Journals joins NumFOCUS Sponsored Projects

NumFOCUS is pleased to welcome the Open Journals as a fiscally sponsored project. Open Journals is a collection of open source, open access journals. The team behind Open Journals believes that code review and high-quality, reusable software are a critical-but often overlooked-part of academic research. The primary goal of Open Journals is to provide venues […]

22 Sep 2017 2:00pm GMT

Georges Dubus: Share scripts that have dependencies with Nix

Share scripts that have dependencies with Nix

Writing and sharing scripts is essential to productivity in development teams. But if you want the script to work everywhere, you can only depend on the tools that are everywhere. That usually means not using the newer features of a language, and not using libraries. In this article, I explain how I use Nix to write scripts that can run everywhere, regardless on what they depend on.

In my work as a software engineer, I often write small scripts to help with administrative actions or provide some nice view of what we are working on. From time to time, one of these scripts is useful enough that I want to share it with my colleagues.

But sharing the scripts comes with a challenge. I write my scripts in Python because it is a great language for that kind of things. But the best parts are outside of the standard library. As a result, to share my scripts, I also have to write a documentation explaining how to get the right version of python, create a python virtual environment, activate it, install the dependencies, and run the script. That's not worth the hassle, and my colleagues won't bother following the complex set of instructions for a script.

Recently, I've started using Nix to make self-contained scripts, and it's been working pretty well.

A script that can download its dependencies

For example, some of the things I like using in python scripts are:

  • the new string formatting syntax in Python 3.6,
  • the requests package for anything that touches HTTP
  • docopt for argument parsing.

My scripts will usually look like:

#! /usr/bin/env nix-shell
#! nix-shell -i python -p python36 python36Packages.requests2 python36Packages.docopt -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz
# You'll need nix to automatically download the dependencies: `curl https://nixos.org/nix/install | sh`
"""
Some script

Usage:
  some_script.py <user> <variant> [--api=<api>]
  some_script.py -h | --help

Arguments:
  <user>        format: 42
  <variant>     format: 2

Options:
  -h --help     Show this screen.
  --api=<api>   The api instance to talk to [default: the-api.internal]
"""
import requests
from docopt import docopt

def doit(user, variant, api):
    # The actual thing I want to do
    pass

if __name__ == '__main__':
    arguments = docopt(__doc__)
    doit(arguments['<user>'], arguments['<variant>'], arguments['--api'])

Now, anybody can run the script with ./some_script.py, which will download the dependencies and run the script with them. They'll need to install nix if they don't have it yet (with curl https://nixos.org/nix/install | sh).

What actually happens there?

The first line of the script is a shebang. It tells the system to run the script with nix-shell, one of the tools provided by Nix.

The second line specifies how to invoke nix-shell:

  • -i python: this argument means we want nix-shell to run the script with the python command (-i stands for interpreter).
  • -p python36 python36Packages.requests2 python36Packages.docopt: this argument is the list of the nix packages that my script is going to use. Here, I want python 3.6, and the python packages requests and docopt. I usually use the Nix packages page to find the name of the dependencies I want.
  • -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz: the version of nix I want to use for this script. This ensures people running the script in the future will get compatible versions of the dependencies.

When I run the script, nix-shell starts by downloading https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz if it has not done so recently. This is the definition of all the nix packages. Then, nix-shell looks into this file to know how to download the packages I want and their dependencies. Finally, it runs my script in an environment where the dependencies are available.

Try it now

This technique allows coworkers to share scripts written in different technologies without the cost of teaching everybody how to install dependencies and run scripts in each language.

Try it yourself right now with the example script from earlier. Make sure you have Nix (or install it with curl https://nixos.org/nix/install | sh), get the script, and run it without caring about the dependencies!

22 Sep 2017 1:00pm GMT

Georges Dubus: Share scripts that have dependencies with Nix

Share scripts that have dependencies with Nix

Writing and sharing scripts is essential to productivity in development teams. But if you want the script to work everywhere, you can only depend on the tools that are everywhere. That usually means not using the newer features of a language, and not using libraries. In this article, I explain how I use Nix to write scripts that can run everywhere, regardless on what they depend on.

In my work as a software engineer, I often write small scripts to help with administrative actions or provide some nice view of what we are working on. From time to time, one of these scripts is useful enough that I want to share it with my colleagues.

But sharing the scripts comes with a challenge. I write my scripts in Python because it is a great language for that kind of things. But the best parts are outside of the standard library. As a result, to share my scripts, I also have to write a documentation explaining how to get the right version of python, create a python virtual environment, activate it, install the dependencies, and run the script. That's not worth the hassle, and my colleagues won't bother following the complex set of instructions for a script.

Recently, I've started using Nix to make self-contained scripts, and it's been working pretty well.

A script that can download its dependencies

For example, some of the things I like using in python scripts are:

  • the new string formatting syntax in Python 3.6,
  • the requests package for anything that touches HTTP
  • docopt for argument parsing.

My scripts will usually look like:

#! /usr/bin/env nix-shell
#! nix-shell -i python -p python36 python36Packages.requests2 python36Packages.docopt -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz
# You'll need nix to automatically download the dependencies: `curl https://nixos.org/nix/install | sh`
"""
Some script

Usage:
  some_script.py <user> <variant> [--api=<api>]
  some_script.py -h | --help

Arguments:
  <user>        format: 42
  <variant>     format: 2

Options:
  -h --help     Show this screen.
  --api=<api>   The api instance to talk to [default: the-api.internal]
"""
import requests
from docopt import docopt

def doit(user, variant, api):
    # The actual thing I want to do
    pass

if __name__ == '__main__':
    arguments = docopt(__doc__)
    doit(arguments['<user>'], arguments['<variant>'], arguments['--api'])

Now, anybody can run the script with ./some_script.py, which will download the dependencies and run the script with them. They'll need to install nix if they don't have it yet (with curl https://nixos.org/nix/install | sh).

What actually happens there?

The first line of the script is a shebang. It tells the system to run the script with nix-shell, one of the tools provided by Nix.

The second line specifies how to invoke nix-shell:

  • -i python: this argument means we want nix-shell to run the script with the python command (-i stands for interpreter).
  • -p python36 python36Packages.requests2 python36Packages.docopt: this argument is the list of the nix packages that my script is going to use. Here, I want python 3.6, and the python packages requests and docopt. I usually use the Nix packages page to find the name of the dependencies I want.
  • -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz: the version of nix I want to use for this script. This ensures people running the script in the future will get compatible versions of the dependencies.

When I run the script, nix-shell starts by downloading https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz if it has not done so recently. This is the definition of all the nix packages. Then, nix-shell looks into this file to know how to download the packages I want and their dependencies. Finally, it runs my script in an environment where the dependencies are available.

Try it now

This technique allows coworkers to share scripts written in different technologies without the cost of teaching everybody how to install dependencies and run scripts in each language.

Try it yourself right now with the example script from earlier. Make sure you have Nix (or install it with curl https://nixos.org/nix/install | sh), get the script, and run it without caring about the dependencies!

22 Sep 2017 1:00pm GMT

Import Python: #143 - Build Book Recommender System, Terrible Python Error Message, Free CI eBook and more

Worthy Read

This free reference guide will take you back to the basics. You'll find visuals and definitions on key concepts and questions you need to answer about your teams to determine your readiness for continuous delivery. Download and share with your team.
GoCD
,
advert

Building recommender systems today requires specialized expertise in analytics, machine learning and software engineering, and learning new skills and tools is difficult and time-consuming. In this post, we will start from scratch, covering some basic fundamental techniques and implementations in Python. In the future posts, we will cover more sophisticated methods such as content-based filtering and collaborative based filtering.
machine learning

What should be a really simple function has turned into a fifty-line gargantuan that's too hard to read properly because of the sheer number of lines. The culprit; a seriously long if-elsif-else chain. But not to fear; there's a better way!
core-python

TypeError: object() takes no parameters
core-python

Modern microprocessors are a very complex piece of machinery with a lot of different parts. I've learned assembler and microprocessors in my undergraduate course some years ago but, as I've been working on higher level software languages for quite some time now, I forgot many things. Lately I decided to revisit some of the topics on microprocessors and to build a very simple model of a microprocessor in Python for education purposes (I work as CS professor). You can find the final source code and some samples at https://github.com/joaoventura/simproc.
simulator

Plotly has a new member of the Plotly.js chart family: The Sankey diagram. Jupyter notebook is at above the footnote of the blogpost.
charts

Integrations, simplified: Integrate your SaaS app with others apps in days, not months. Get started for free!
advert

I got my hands on a dataset on traffic accidents in Metro Manila early this year, and decided to enter the realm of data science with a simple cleaning and visualization project. The primary goal: supplement the dataset's human readable location data with geographic coordinates.
data science

Have you ever noticed that Python supports integers of any size? Here is a quick glance at it.
cpython

In the last post, we saw how we could use simple correlational techniques to create a measure of similarity between the books' users based on their rating records. In this post, we will explain how you can use those same sort of similarity metrics to recommend books to a book's readers.
machine learning

I wanted to make a short post on using Slack's incoming webhooks feature to post messages to Slack using the human-friendly Requests API in Python3.
slack

We help companies like Airbnb, Pfizer, and Artsy find great developers. Let us find your next great hire. Get started today.
advert

Code packages available in PyPI contained modified installation scripts.
pypi

In this short blogpost, I will explore few topics to illustrate the interactivity of the jupyter environment and the python availability of high quality libraries in the ecosystem.
jupyter

The term desktop notifications refer to a graphical control element that communicates certain events to the user without forcing them to react to this notification immediately. In other words, it is a simple application which produces a notification message in form of a pop-up message on desktop.
desktop app

Geospatial analysis applies statistical analysis to data that has geographical or geometrical components. In this tutorial, we'll use Python to learn the basics of acquiring geospatial data, handling it, and visualizing it. More specifically, we'll do some interactive visualizations of the United States!
geo

The current blog deals with the case how to implement web analytics in Python. I am enthusiastic about the options that are available inside Google Analytics. Google Analytics has a rich variety of metrics and dimensions available. It has a good visualization and an intuitive Graphic User Interface (GUI). However, in certain situations it makes sense to automate webanalytics and add advanced statistics and visualizations. In the current blog, I will show how to do that using Python.
analytics

Transit networks with higher frequency and shorter waiting times will yield a more reliable and empowering experience for passengers than those with lower frequency and longer waiting times.
geo

I needed a quick way to remotely perform system operations: adjusting & muting the system volume, screen brightness, and putting the display to sleep. After exploring several options, I found none of them to be viable for me, and thus, I set out to develop my own solution.
macOS

In the example below, I've demonstrated how this can be done using Python in a way that doesn't require deep knowledge of CUDA and its intricacies. For this example, I suggest using the Anaconda Python distribution, which makes managing different Python environments a breeze. Follow the download and setup instructions for Anaconda as given here for your specific operating system.
GPU

Pokemon

How to share code snippets from command line?
github
,
gist

Some time ago we decided to move part of our logic to celery. Several tasks was wrote and in one of them bash script was called, and this task never applied correct: first half of script was executed and then halted. I found that script running in processes, and can't understand why it just stoped. Then I set some breakpoints in bash script and realised, that script stops at
celery

From dummy variables to Deep category embedding and Cat2vec?-?Part 1 (Basic Methods).
machine learning

You can join two GeoPandas GeoDataFrames through conventional means with merge, but you can also use sjoin to capitalize on the spatial relationship between two frames.
pandas

text processing


Projects

face-alignment - 165 Stars, 25 Fork
2D and 3D Face alignment library build using pytorch

MetaDockers - 44 Stars, 8 Fork
Responsible for visualization the vulhub or docker.

mygf-instagram - 34 Stars, 1 Fork
Like posts of my girlfriend's Instagram using web api, don't need any api key and access tokens just login and password

graphscii - 3 Stars, 0 Fork
Print ASCII graphs in the terminal.

xlsx2csv_gui - 2 Stars, 0 Fork
Convert excel xlsx file's table to csv file, A GUI application on top of python/pyqt and other opensource softwares.

22 Sep 2017 12:14pm GMT

Import Python: #143 - Build Book Recommender System, Terrible Python Error Message, Free CI eBook and more

Worthy Read

This free reference guide will take you back to the basics. You'll find visuals and definitions on key concepts and questions you need to answer about your teams to determine your readiness for continuous delivery. Download and share with your team.
GoCD
,
advert

Building recommender systems today requires specialized expertise in analytics, machine learning and software engineering, and learning new skills and tools is difficult and time-consuming. In this post, we will start from scratch, covering some basic fundamental techniques and implementations in Python. In the future posts, we will cover more sophisticated methods such as content-based filtering and collaborative based filtering.
machine learning

What should be a really simple function has turned into a fifty-line gargantuan that's too hard to read properly because of the sheer number of lines. The culprit; a seriously long if-elsif-else chain. But not to fear; there's a better way!
core-python

TypeError: object() takes no parameters
core-python

Modern microprocessors are a very complex piece of machinery with a lot of different parts. I've learned assembler and microprocessors in my undergraduate course some years ago but, as I've been working on higher level software languages for quite some time now, I forgot many things. Lately I decided to revisit some of the topics on microprocessors and to build a very simple model of a microprocessor in Python for education purposes (I work as CS professor). You can find the final source code and some samples at https://github.com/joaoventura/simproc.
simulator

Plotly has a new member of the Plotly.js chart family: The Sankey diagram. Jupyter notebook is at above the footnote of the blogpost.
charts

Integrations, simplified: Integrate your SaaS app with others apps in days, not months. Get started for free!
advert

I got my hands on a dataset on traffic accidents in Metro Manila early this year, and decided to enter the realm of data science with a simple cleaning and visualization project. The primary goal: supplement the dataset's human readable location data with geographic coordinates.
data science

Have you ever noticed that Python supports integers of any size? Here is a quick glance at it.
cpython

In the last post, we saw how we could use simple correlational techniques to create a measure of similarity between the books' users based on their rating records. In this post, we will explain how you can use those same sort of similarity metrics to recommend books to a book's readers.
machine learning

I wanted to make a short post on using Slack's incoming webhooks feature to post messages to Slack using the human-friendly Requests API in Python3.
slack

We help companies like Airbnb, Pfizer, and Artsy find great developers. Let us find your next great hire. Get started today.
advert

Code packages available in PyPI contained modified installation scripts.
pypi

In this short blogpost, I will explore few topics to illustrate the interactivity of the jupyter environment and the python availability of high quality libraries in the ecosystem.
jupyter

The term desktop notifications refer to a graphical control element that communicates certain events to the user without forcing them to react to this notification immediately. In other words, it is a simple application which produces a notification message in form of a pop-up message on desktop.
desktop app

Geospatial analysis applies statistical analysis to data that has geographical or geometrical components. In this tutorial, we'll use Python to learn the basics of acquiring geospatial data, handling it, and visualizing it. More specifically, we'll do some interactive visualizations of the United States!
geo

The current blog deals with the case how to implement web analytics in Python. I am enthusiastic about the options that are available inside Google Analytics. Google Analytics has a rich variety of metrics and dimensions available. It has a good visualization and an intuitive Graphic User Interface (GUI). However, in certain situations it makes sense to automate webanalytics and add advanced statistics and visualizations. In the current blog, I will show how to do that using Python.
analytics

Transit networks with higher frequency and shorter waiting times will yield a more reliable and empowering experience for passengers than those with lower frequency and longer waiting times.
geo

I needed a quick way to remotely perform system operations: adjusting & muting the system volume, screen brightness, and putting the display to sleep. After exploring several options, I found none of them to be viable for me, and thus, I set out to develop my own solution.
macOS

In the example below, I've demonstrated how this can be done using Python in a way that doesn't require deep knowledge of CUDA and its intricacies. For this example, I suggest using the Anaconda Python distribution, which makes managing different Python environments a breeze. Follow the download and setup instructions for Anaconda as given here for your specific operating system.
GPU

Pokemon

How to share code snippets from command line?
github
,
gist

Some time ago we decided to move part of our logic to celery. Several tasks was wrote and in one of them bash script was called, and this task never applied correct: first half of script was executed and then halted. I found that script running in processes, and can't understand why it just stoped. Then I set some breakpoints in bash script and realised, that script stops at
celery

From dummy variables to Deep category embedding and Cat2vec?-?Part 1 (Basic Methods).
machine learning

You can join two GeoPandas GeoDataFrames through conventional means with merge, but you can also use sjoin to capitalize on the spatial relationship between two frames.
pandas

text processing


Projects

face-alignment - 165 Stars, 25 Fork
2D and 3D Face alignment library build using pytorch

MetaDockers - 44 Stars, 8 Fork
Responsible for visualization the vulhub or docker.

mygf-instagram - 34 Stars, 1 Fork
Like posts of my girlfriend's Instagram using web api, don't need any api key and access tokens just login and password

graphscii - 3 Stars, 0 Fork
Print ASCII graphs in the terminal.

xlsx2csv_gui - 2 Stars, 0 Fork
Convert excel xlsx file's table to csv file, A GUI application on top of python/pyqt and other opensource softwares.

22 Sep 2017 12:14pm GMT

PyCharm: PyCharm Edu: Tips & Tricks for Most Efficient Learning, Part II

September feels just right to start learning, or continue with even more effort and joy. And today feels just right for the next post in our series covering learning methods and tips & tricks designed to help you to learn more effectively and make you more excited about learning Python with PyCharm Edu!

Last time we posted about how to make your IDE suit your needs and preferences, and help you stay focused and avoid distraction. Today we're going to focus more on the process of writing code.

Learn faster with a smart code editor

As you learn to code, it is encouraging to have an intelligent assistant right inside your IDE. Language-specific syntax & error highlighting are there to help you avoid code mistakes. You can use code formatting capabilities to learn how to style your code. And you won't get stuck with your Python coding with code completion and quick documentation at hand.

Use code completion
Learn with documentation
Explore the code
Fix code errors
Style your code

Please note that we use the default Mac OS keyboard shortcut scheme (Mac OS X 10.5+) in this blog post. For more about keyboard shortcuts, please see the previous blog post in this series.

Use code completion

Smart autocompletion is an essential feature of a professional code editor, which helps you save time and avoid misprints. It is equally powerful for learning because you can use it to find the right variable, function, parameter, or any code construct. Start typing or simply press Control-Space to get a list of suggested options:Code completion

Code snippets, or live templates, can also be a handy tool. They not only speed up your coding, but help you learn new language syntax as well. Use Command-J to see the full list of code snippets available. Start typing to find the one you want to add, and then use Tab to fill in the blanks:Live templates

Learn with documentation

Reading documentation is a must when learning a new programming language. PyCharm Edu makes this process as seamless as possible.

Put a cursor on a class, method, variable, or function you want to read about, and then press F1 to open the quick documentation pop-up:documentation_1

Use F1 once more to switch to the tool window mode:documantation_2

You can click hyperlinks to find more details. For example, click int to read about the int class:documantation3_1documentation_3

Use the Back and Forward icons to get back to the documentation pages you visited recently:documentation_4documentation_5

Save your time and effort with the special Auto-update from Source icon. When switched on, it lets you read the documentation in parallel with reading your code. Go through a code fragment and read about all the code constructions you put your cursor on, without any additional shortcuts or mouse clicks:Documentation

Explore the code

Programming is much more about understanding and reading existing code than about writing new code. That's why we develop so many code navigation features in our IDEs. While learning, it is also quite important to see your code as a whole and understand how it works.

When you use any variable, function, or method and need a tip on how it works, let the code speak for itself. Press Command-B to go to the declaration of an entity; use it one more time on the declaration to see the list of usages:Navigation

Another quick way to look at the declaration is to use the pop-up (with Command-Y):quick_definition

Fix code errors

Mistakes are an essential part of successful learning. They can also be very annoying and drain your motivation. The IDE helps you not get stuck in code errors, by alerting you to the most common ones and suggesting solutions.

Take a look at the code below. The red squiggly line code under task means there's something wrong with this statement:code_errors

Put the cursor on the highlighted code to see what the issue is. Then, click the red bulb or press Option-Enter to see the list of suggested fixes. Choose the right one to fix your code:Intention actions

It seems there is still something wrong with the file, because there's a red icon at the top right:code_status_red

Click the red stripe to navigate to the line with the error and fix it:code_status_error

Now everything is fixed:code_status_green

Style your code

It is important not only to write the right code, but also to write your code right. Try to follow a code style so your code is easy to read and understand. You'll thank yourself later!

Invoke Option-Command-L and let the IDE reformat your code automatically:
Code style


There we have it. Let us know how you like these features! Stay tuned for the next portion of tips & tricks for more efficient learning. Share your feedback here in the comments or report your findings on YouTrack, to help us improve PyCharm Edu.

-
Your PyCharm Edu Team

22 Sep 2017 11:45am GMT

PyCharm: PyCharm Edu: Tips & Tricks for Most Efficient Learning, Part II

September feels just right to start learning, or continue with even more effort and joy. And today feels just right for the next post in our series covering learning methods and tips & tricks designed to help you to learn more effectively and make you more excited about learning Python with PyCharm Edu!

Last time we posted about how to make your IDE suit your needs and preferences, and help you stay focused and avoid distraction. Today we're going to focus more on the process of writing code.

Learn faster with a smart code editor

As you learn to code, it is encouraging to have an intelligent assistant right inside your IDE. Language-specific syntax & error highlighting are there to help you avoid code mistakes. You can use code formatting capabilities to learn how to style your code. And you won't get stuck with your Python coding with code completion and quick documentation at hand.

Use code completion
Learn with documentation
Explore the code
Fix code errors
Style your code

Please note that we use the default Mac OS keyboard shortcut scheme (Mac OS X 10.5+) in this blog post. For more about keyboard shortcuts, please see the previous blog post in this series.

Use code completion

Smart autocompletion is an essential feature of a professional code editor, which helps you save time and avoid misprints. It is equally powerful for learning because you can use it to find the right variable, function, parameter, or any code construct. Start typing or simply press Control-Space to get a list of suggested options:Code completion

Code snippets, or live templates, can also be a handy tool. They not only speed up your coding, but help you learn new language syntax as well. Use Command-J to see the full list of code snippets available. Start typing to find the one you want to add, and then use Tab to fill in the blanks:Live templates

Learn with documentation

Reading documentation is a must when learning a new programming language. PyCharm Edu makes this process as seamless as possible.

Put a cursor on a class, method, variable, or function you want to read about, and then press F1 to open the quick documentation pop-up:documentation_1

Use F1 once more to switch to the tool window mode:documantation_2

You can click hyperlinks to find more details. For example, click int to read about the int class:documantation3_1documentation_3

Use the Back and Forward icons to get back to the documentation pages you visited recently:documentation_4documentation_5

Save your time and effort with the special Auto-update from Source icon. When switched on, it lets you read the documentation in parallel with reading your code. Go through a code fragment and read about all the code constructions you put your cursor on, without any additional shortcuts or mouse clicks:Documentation

Explore the code

Programming is much more about understanding and reading existing code than about writing new code. That's why we develop so many code navigation features in our IDEs. While learning, it is also quite important to see your code as a whole and understand how it works.

When you use any variable, function, or method and need a tip on how it works, let the code speak for itself. Press Command-B to go to the declaration of an entity; use it one more time on the declaration to see the list of usages:Navigation

Another quick way to look at the declaration is to use the pop-up (with Command-Y):quick_definition

Fix code errors

Mistakes are an essential part of successful learning. They can also be very annoying and drain your motivation. The IDE helps you not get stuck in code errors, by alerting you to the most common ones and suggesting solutions.

Take a look at the code below. The red squiggly line code under task means there's something wrong with this statement:code_errors

Put the cursor on the highlighted code to see what the issue is. Then, click the red bulb or press Option-Enter to see the list of suggested fixes. Choose the right one to fix your code:Intention actions

It seems there is still something wrong with the file, because there's a red icon at the top right:code_status_red

Click the red stripe to navigate to the line with the error and fix it:code_status_error

Now everything is fixed:code_status_green

Style your code

It is important not only to write the right code, but also to write your code right. Try to follow a code style so your code is easy to read and understand. You'll thank yourself later!

Invoke Option-Command-L and let the IDE reformat your code automatically:
Code style


There we have it. Let us know how you like these features! Stay tuned for the next portion of tips & tricks for more efficient learning. Share your feedback here in the comments or report your findings on YouTrack, to help us improve PyCharm Edu.

-
Your PyCharm Edu Team

22 Sep 2017 11:45am GMT

Catalin George Festila: The python-vlc python module.

The python module for vlc is named python-vlc.
This python module let you to test libvlc API like the VLC video player.
You can install it easy with pip python tool.

C:\Python27\Scripts>pip2.7.exe install python-vlc
Collecting python-vlc
Downloading python-vlc-1.1.2.tar.gz (201kB)
100% |################################| 204kB 628kB/s
Installing collected packages: python-vlc
Running setup.py install for python-vlc ... done
Successfully installed python-vlc-1.1.2

Let's see a simple example with this python module:

import os
import sys
import vlc
import pygame

def call_vlc(self, player):

player.get_fps()
player.get_time()

if len( sys.argv )< 2 or len( sys.argv )> 3:
print 'Help: python vlc_001.py your_video.mp4'
else:
pygame.init()
screen = pygame.display.set_mode((800,600),pygame.RESIZABLE)
pygame.display.get_wm_info()
pygame.display.get_driver()


# get path to movie specified as command line argument
movie = os.path.expanduser(sys.argv[1])
# see if movie is accessible
if not os.access(movie, os.R_OK):
print('Error: %s wrong read file: ' % movie)
sys.exit(1)

# make instane of VLC and create reference to movie.
vlcInstance = vlc.Instance()
media = vlcInstance.media_new(movie)

# make new instance of vlc player
player = vlcInstance.media_player_new()

# start with a callback
em = player.event_manager()
em.event_attach(vlc.EventType.MediaPlayerTimeChanged, \
call_vlc, player)

# set pygame window id to vlc player
win_id = pygame.display.get_wm_info()['window']
if sys.platform == "win32": # for Windows
player.set_hwnd(win_id)

# load movie into vlc player instance
player.set_media(media)

# quit pygame mixer to allow vlc full access to audio device
pygame.mixer.quit()

# start movie play
player.play()

while player.get_state() != vlc.State.Ended:
for event in pygame.event.get():
if event.type == pygame.QUIT:
sys.exit(2)

The base of this python script is to make a instance of vlc and put into pygame display.
Another simple example:

C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vlc
>>> inst = vlc.Instance()
Warning: option --plugin-path no longer exists.
Warning: option --plugin-path no longer exists.
>>> med = inst.media_new('rain.mp4')
>>> p = med.player_new_from_media()
>>> p.play()
0
>>>

22 Sep 2017 11:13am GMT

Catalin George Festila: The python-vlc python module.

The python module for vlc is named python-vlc.
This python module let you to test libvlc API like the VLC video player.
You can install it easy with pip python tool.

C:\Python27\Scripts>pip2.7.exe install python-vlc
Collecting python-vlc
Downloading python-vlc-1.1.2.tar.gz (201kB)
100% |################################| 204kB 628kB/s
Installing collected packages: python-vlc
Running setup.py install for python-vlc ... done
Successfully installed python-vlc-1.1.2

Let's see a simple example with this python module:

import os
import sys
import vlc
import pygame

def call_vlc(self, player):

player.get_fps()
player.get_time()

if len( sys.argv )< 2 or len( sys.argv )> 3:
print 'Help: python vlc_001.py your_video.mp4'
else:
pygame.init()
screen = pygame.display.set_mode((800,600),pygame.RESIZABLE)
pygame.display.get_wm_info()
pygame.display.get_driver()


# get path to movie specified as command line argument
movie = os.path.expanduser(sys.argv[1])
# see if movie is accessible
if not os.access(movie, os.R_OK):
print('Error: %s wrong read file: ' % movie)
sys.exit(1)

# make instane of VLC and create reference to movie.
vlcInstance = vlc.Instance()
media = vlcInstance.media_new(movie)

# make new instance of vlc player
player = vlcInstance.media_player_new()

# start with a callback
em = player.event_manager()
em.event_attach(vlc.EventType.MediaPlayerTimeChanged, \
call_vlc, player)

# set pygame window id to vlc player
win_id = pygame.display.get_wm_info()['window']
if sys.platform == "win32": # for Windows
player.set_hwnd(win_id)

# load movie into vlc player instance
player.set_media(media)

# quit pygame mixer to allow vlc full access to audio device
pygame.mixer.quit()

# start movie play
player.play()

while player.get_state() != vlc.State.Ended:
for event in pygame.event.get():
if event.type == pygame.QUIT:
sys.exit(2)

The base of this python script is to make a instance of vlc and put into pygame display.
Another simple example:

C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vlc
>>> inst = vlc.Instance()
Warning: option --plugin-path no longer exists.
Warning: option --plugin-path no longer exists.
>>> med = inst.media_new('rain.mp4')
>>> p = med.player_new_from_media()
>>> p.play()
0
>>>

22 Sep 2017 11:13am GMT

Gocept Weblog: Earl Zope II is dead, long live Earl Zope

Narrative version

Once upon the time there was Earl Zope II. His lands where threatened by a huge disaster called "Python 2 sunset". His only chance was to emigrate to the Python 3 wonderland. After a long preparation phase for himself and his courtiers he was able to move to the new land. But the Python 3 wonderland has strict immigration authorities: They only allow "compatible" fellows to get a residency permit. The permission itself has three levels:

Back in his own country Earl Zope II had the official name "Zope2 2.13". Before the beta phase of his immigration he thought that he would have to change the name to something like "Zope2 4.0b1". This is looks ugly and some people protested against this name. With the kind and quick help of Baiju Muthukadan Earl Zope II was able to change his official name to "Zope 4.0b1". Thank you very much Baiju Muthukadan! Earl Zope is very proud of this new shorter name and is heartily thankful for this opportunity.

Technical version

The release of the first Beta version means, that no currently existing features will be removed until the final version. But There will be some new features and many bugfixes.

New Features of Zope version 4.0

Changes

Thanks

A big "Thank you!" to all who made this release possible:

We had a great Zope 4 Phoenix Sprint helping to raise Zope from the ashes! Thank you to everybody who participated in Halle or from remote.

Roadmap

Beta one of Zope is out:

We need the feedback to adapt Zope to the needs in the wild. The current plan is to create new beta releases once a while after implementing features resp. bug fixes or on demand. The current plan is to release a final 4.0 version in fall of 2018. This should allow software projects built upon Zope to migrate there code before the Python 2 sunset in 2020.


22 Sep 2017 8:05am GMT

Gocept Weblog: Earl Zope II is dead, long live Earl Zope

Narrative version

Once upon the time there was Earl Zope II. His lands where threatened by a huge disaster called "Python 2 sunset". His only chance was to emigrate to the Python 3 wonderland. After a long preparation phase for himself and his courtiers he was able to move to the new land. But the Python 3 wonderland has strict immigration authorities: They only allow "compatible" fellows to get a residency permit. The permission itself has three levels:

Back in his own country Earl Zope II had the official name "Zope2 2.13". Before the beta phase of his immigration he thought that he would have to change the name to something like "Zope2 4.0b1". This is looks ugly and some people protested against this name. With the kind and quick help of Baiju Muthukadan Earl Zope II was able to change his official name to "Zope 4.0b1". Thank you very much Baiju Muthukadan! Earl Zope is very proud of this new shorter name and is heartily thankful for this opportunity.

Technical version

The release of the first Beta version means, that no currently existing features will be removed until the final version. But There will be some new features and many bugfixes.

New Features of Zope version 4.0

Changes

Thanks

A big "Thank you!" to all who made this release possible:

We had a great Zope 4 Phoenix Sprint helping to raise Zope from the ashes! Thank you to everybody who participated in Halle or from remote.

Roadmap

Beta one of Zope is out:

We need the feedback to adapt Zope to the needs in the wild. The current plan is to create new beta releases once a while after implementing features resp. bug fixes or on demand. The current plan is to release a final 4.0 version in fall of 2018. This should allow software projects built upon Zope to migrate there code before the Python 2 sunset in 2020.


22 Sep 2017 8:05am GMT

Gocept Weblog: Move documentation from pythonhosted.org to readthedocs.io

Today we migrated the documentation of zodb.py3migrate from pythonhosted.org to zodbpy3migrate.readthedocs.io.

This requires a directory - for this example I name it redir - containing a file named index.html with the following content:

<html>
<head>
 <title>zodb.py3migrate</title>
 <meta http-equiv="refresh"
       content="0; url=http://zodbpy3migrate.rtfd.io" />
</head>
<body>
  <p>
    <a href="http://zodbpy3migrate.rtfd.io">
      Redirect to zodbpy3migrate.rtfd.io
    </a>
  </p>
</body>
</html>

To upload it to pythonhosted.org I called:

py27 setup.py upload_docs --upload-dir=redir

Now pythonhosted.org/zodb.py3migrate points to read the docs.

Credits: The HTML was taken from the Trello board of the avocado-framework.


22 Sep 2017 6:28am GMT

Gocept Weblog: Move documentation from pythonhosted.org to readthedocs.io

Today we migrated the documentation of zodb.py3migrate from pythonhosted.org to zodbpy3migrate.readthedocs.io.

This requires a directory - for this example I name it redir - containing a file named index.html with the following content:

<html>
<head>
 <title>zodb.py3migrate</title>
 <meta http-equiv="refresh"
       content="0; url=http://zodbpy3migrate.rtfd.io" />
</head>
<body>
  <p>
    <a href="http://zodbpy3migrate.rtfd.io">
      Redirect to zodbpy3migrate.rtfd.io
    </a>
  </p>
</body>
</html>

To upload it to pythonhosted.org I called:

py27 setup.py upload_docs --upload-dir=redir

Now pythonhosted.org/zodb.py3migrate points to read the docs.

Credits: The HTML was taken from the Trello board of the avocado-framework.


22 Sep 2017 6:28am GMT

Sandipan Dey: Some more Social Network Analysis with Python: Centrality, PageRank/HITS, Random Network Generation Models, Link Prediction

In this article, some more social networking concepts will be illustrated with a few problems. The problems appeared in the programming assignments in the coursera course Applied Social Network Analysis in Python. The descriptions of the problems are taken from the assignments. The analysis is done using NetworkX. The following theory is going to be used to solve … Continue reading Some more Social Network Analysis with Python: Centrality, PageRank/HITS, Random Network Generation Models, Link Prediction

22 Sep 2017 12:02am GMT

Sandipan Dey: Some more Social Network Analysis with Python: Centrality, PageRank/HITS, Random Network Generation Models, Link Prediction

In this article, some more social networking concepts will be illustrated with a few problems. The problems appeared in the programming assignments in the coursera course Applied Social Network Analysis in Python. The descriptions of the problems are taken from the assignments. The analysis is done using NetworkX. The following theory is going to be used to solve … Continue reading Some more Social Network Analysis with Python: Centrality, PageRank/HITS, Random Network Generation Models, Link Prediction

22 Sep 2017 12:02am GMT

21 Sep 2017

feedPlanet Python

Brian Okken: What info do you need to start testing?

I got some feedback today from someone who said they were looking through the pythontesting.net/start-here page and were still a bit confused as to how to start testing. I totally understand that and want to improve it. I want to make the site more friendly to people new to testing. What info would help? (try […]

The post What info do you need to start testing? appeared first on Python Testing.

21 Sep 2017 8:42pm GMT

Brian Okken: What info do you need to start testing?

I got some feedback today from someone who said they were looking through the pythontesting.net/start-here page and were still a bit confused as to how to start testing. I totally understand that and want to improve it. I want to make the site more friendly to people new to testing. What info would help? (try […]

The post What info do you need to start testing? appeared first on Python Testing.

21 Sep 2017 8:42pm GMT

Continuum Analytics Blog: Back to School: Data Science & AI Open New Doors for Students

School is back in session and now is the time students are thinking about their future. When considering options, science-oriented students are likely thinking about what is arguably today's hottest technology: artificial intelligence (AI).

21 Sep 2017 6:40pm GMT

Continuum Analytics Blog: Back to School: Data Science & AI Open New Doors for Students

School is back in session and now is the time students are thinking about their future. When considering options, science-oriented students are likely thinking about what is arguably today's hottest technology: artificial intelligence (AI).

21 Sep 2017 6:40pm GMT

Stack Abuse: Flask vs Django

Flask vs Django

In this article, we will take a look at two of the most popular web frameworks in Python: Django and Flask.

Here, we will be covering how each of these frameworks compares when looking at their learning curves, how easy it is to get started. Next, we'll also be looking at how these two stands against each other with concluding by when to use one of them.

Getting Started

One of the easiest ways to compare two frameworks is by installing them and taking note how easily a user can get started with it, which is exactly what we will do next. We will try setting up Django and Flask on a Linux machine and create an app to see how easy (or difficult) the process is with each one.

Setting up Django

In this section, we will setup Django on an Linux-powered machine. Best way to get started with any Python framework is by using virtual environments. We will install it using pip.

$ sudo apt-get install python3-pip
$ pip3 install virtualenv
$ virtualenv --python=`which python3` ~/.virtualenvs/django_env

Note: If the pip3 command gives you an error, you may need to prefix it with sudo to make it work.

Once we're done setting up our virtual environment, which we've named django_env, we must activate it to start using it:

$ source ~/.virtualenvs/django_env/bin/activate

Once activated, we can finally install Django:

$ pip install Django

Suppose our project is called mysite. Make a new directory and enter it, run following commands:

$ mkdir mysite
$ cd mysite
$ django-admin startproject mysite

If you inspect the resulting project, your directory structure will be shown as:

mysite/  
  manage.py 
  mysite/
    __init__.py 
    settings.py 
    urls.py 
    wsgi.py

Let's take a look at what is significant about each of the directories and files that were created.

From here, we can actually run the app using the manage.py tool. The following command does some system checks, checks for database migrations, and some other things before actually running your server:

$ python manage.py runserver

Performing system checks...

System check identified no issues (0 silenced).

You have unapplied migrations; your app may not work properly until they are applied.  
Run 'python manage.py migrate' to apply them.

September 20, 2017 - 15:50:53  
Django version 1.11, using settings 'mysite.settings'  
Starting development server at http://127.0.0.1:8000/  
Quit the server with CONTROL-C.  

Note: Running your server in this way is meant for development only, and not production environments.

To check out your app, head to http://localhost:8000/, where you should see a page saying "It worked!".

But wait, you're still not done! To actually create any pages/functionality in your site, you need to create an app within your project. But why do you need an app? In Django, apps are web applications that do something, which could be a blog, a forum, or a commenting system. The project is a collection of your apps, as well as configuration for the apps and entire website.

So, to create your app, move in to your project directory and run the following command:

$ cd mysite
$ python manage.py startapp myapp

This will create another directory structure where you can actually manage your models, views, etc.

manage.py  
myapp/  
  __init__.py
  admin.py
  apps.py
  migrations/
  models.py
  tests.py
  views.py
mysite/  
  __init__.py 
  settings.py 
  urls.py 
  wsgi.py

From here, you need to set up your views in views.py and URL routing in urls.py, which we'll save for another tutorial.

But you get the point, right? It takes a few commands and quite a few files to get your Django project up and running.

Setting up Flask

Just like Django, we will use a virtual environment with Flask as well. So the commands for activating a virtual environment will remain the same as before. After that, instead of installing Django, we'll install Flask instead.

$ pip install Flask

Once the installation completes, we can start creating our Flask application. Now, unlike Django, Flask doesn't have a complicated directory structure. The structure of your Flask project is entirely up to you.

Borrowing an example from the Flask homepage, you can create a runnable Flask app from just a single file:

from flask import Flask  
app = Flask(__name__)

@app.route("/")
def hello():  
    return "Hello World!"

And running the app is about as easy as setting it up:

$ FLASK_APP=hello.py flask run
 * Running on http://localhost:5000/

Visiting the URL http://localhost:5000/ should display the text "Hello World!" in your browser.

I'd encourage you to look for some sample apps on the Flask homepage to learn more. Learning by example is one of the best ways to get up and running quickly.

The framework that "wins" this area is really up to your needs and experience. Django may be more favorable for beginners since it makes decisions for you (i.e. how to structure your app), whereas in Flask you need to handle this yourself.

On the other hand, Flask is simpler to get running since it requires very little to get going. An entire Flask app can be composed from a single file. The trade-offs really depend no what you need most.

Learning Curve

Regarding learning curve, as we saw in the last section with Flask, it was very easy to get started. The app doesn't require a complicated directory structure where you needed to remember which directory/file does what. Instead, you can add files and directories as you go according to your usage. This is what Flask is about, as a micro-framework for web development.

Django, on the other hand, has a bit higher learning curve since it is more "picky" about how things are set up and work. Because of this, you need to take more time learning how to compose modules and work within the confines of the framework.

This isn't all bad though, since this allows you to easily plug in 3rd party components in to your app without having to do any work integrating them.

Employability

Which of these frameworks will help you land a job? For many developers, this is one of the more important question regarding certain libraries and frameworks: which will help me get hired?

Django has quite a few large companies on its resume, which is because many companies that use Python for web development tend to use (or at least started off with) Django to power their site. Django, being a full-fledged framework, is often used early on in development because you get much more resources and power with it out of the box.

Here are just a few companies that use (or have used) Django for their sites:

Flask is a bit harder to gauge here, mostly because of the way it is used. Flask tends to be used more for microservices, which makes it harder to tell which companies are using it. Plus, companies with a microservice architecture are less likely to say their service is "powered by Flask" since they likely have many services potentially using many different frameworks.

There are, however, hints out there of who uses Flask based on job postings, tech talks, blog posts, etc. From those, we know that the following companies have used Flask somewhere in their backend infrastructure:

While Django may be more popular among companies, Flask is arguably more common among the more tech-focused companies as they are more likely to use microservices, and therefore micro-frameworks like Flask.

Project Size and Scope

Our comparison of each framework can become very subjective thanks to many different factors, like project scope, developer experience, type of project, etc. If the project is fairly small and it doesn't need all of the overhead that Django comes with, then Flask is the ideal choice to get started and get something done very quickly.

However, if the project is larger in duration and scope, then Django is likely the way to go as it already includes much of what you need. This basically means that many common components of a web-service/website either already comes with Django, or it is already available through 3rd party open source software. In some cases you can just create a Django project, plug in a bunch of components, create your views/templates, and you're done.

While we do praise Django for its extensibility, we can't ignore that Flask does have some extensions of its own. While they aren't quite as big in scope as Django (and many of these extensions come standard in Django), it's a step in the right direction.

Django's add-on components can be as big as a blog add-on, to as small as small middleware input validation. Most of Flask's extensions are small middleware components, which is still better than nothing and very helpful, considering the average size of Flask projects.

Limitations

Every piece of tech has its problems, and these frameworks are no different. So before you choose which to use, you might want to know what disadvantages each has, which we'll be talking about in this section.

Django

So, what are the aspects of Django that work against it to be selected as your framework of choice?

Django is a very large project. Once a developer, especially beginners, starts learning Django, it's easy for them to get lost in the source code, the built-in features, and components it provides, without even using them in an app.

Django is a fairly large framework to deploy for simple use-cases, as it hides much of the control from you. If you want to use something that isn't "standard" in Django, then you have to put in some extra work to do so.

Understanding components in Django can be a little difficult and tricky at times and can lead to tough decisions, like deciding if an existing component will work for your use-case, or if it'll end up causing you more work than it is worth.

Flask

Now that we've seen some of the problems with Django, let's not forget about Flask. Since the Flask framework is so small, there isn't a lot to complain about. Well, except for that fact right there: It's so small.

Flask is a micro-framework, which means it only provides the bare-bones functionality to get you started. This doesn't mean it can't be powerful and can't scale, it just means that you'll have to create much of the functionality of your service yourself. This means you'll need to handle integrating your database, data validation, file serving, etc.

While this could be considered an advantage to those who want control over everything, it also means it'll take you longer to get set up with a fully-functional website.

Choosing Flask or Django

While it's easy to talk about what each framework does and doesn't do, let's try and make a more direct comparison of each, which we'll do in this section.

When simplicity is a factor, Flask is the way to go. It allows much more control over your app and lets you decide how you want to implement things in a project. In contrast to this, Django provides a more inclusive experience, such as providing a default admin panel for your data, an ORM on top of your database, and protection against things like SQL injection, cross-site scripting, CSRF, etc.

If you put a lot of emphasis on community support, then Django is probably better in this regard given its history. It has been around since 2005, whereas Flask was created in 2010. At the time of writing this article, Django has about 3.5x more questions/answers on Stack Overflow than Flask (about 2600 Django questions to Flask's 750).

The Flask framework is relatively lightweight. In fact, it's almost 2.5x smaller than Django in terms of amount of code. That's a big difference, especially if you need to understand the inner-workings of your web framework. In this aspect, Flask will be much easier to read and understand for most developers.

Flask should be selected for development if you need complete control over your app, which ORM you want to use, which database you need to integrate with excellent opportunities to learn more about web services. Django, on the other hand, is better when there is a more clear path to creating what you want, or you're creating something that has been done before. For example, a blog would be a good use-case for Django.

Learn More

Want to learn more about either of these frameworks? There are quite a few resources out there. Here are a few courses that I've found to be pretty helpful, and will get you up to speed much quicker:

Flask vs Django
Python and Django Full Stack Web Developer Bootcamp

Flask vs Django
REST APIs with Flask and Python

Otherwise you can also get a great start by visiting each framework's respective websites:

Either way, the most important thing is you actually try them out, work through some examples, and decide on your own which is best for you.

Conclusion

In this article, we compared the two web frameworks, Django and Flask, by looking at their different properties and setting up a simple "Hello World!" app with each one.

You may find that if you're new to web development and decide to learn Django, it may take you a bit longer to truly understand what all of the underlying components do, and how to change them to actually do what you want. But there are many positives as well, and once you become proficient with Django, it'll end up saving you a lot of time in the end, given its huge list of components and vast community support.

A more advanced comparison for any frameworks can be done only with advanced use cases and scenarios. Just know that you can't really go wrong with either one, and learning either will set you up well for finding a job.

If you need a recommendation, then I'd personally go with Flask. By learning a framework that doesn't hide so much stuff from you, you can learn much, much more. Once you have a better understanding of the core concepts of web development and HTTP, you can start to use add-ons that abstract this away from you. But having that solid foundation of understanding is more important early on, in my opinion.

Which framework do you use, and why? Let us know in the comments!

21 Sep 2017 3:52pm GMT

Stack Abuse: Flask vs Django

Flask vs Django

In this article, we will take a look at two of the most popular web frameworks in Python: Django and Flask.

Here, we will be covering how each of these frameworks compares when looking at their learning curves, how easy it is to get started. Next, we'll also be looking at how these two stands against each other with concluding by when to use one of them.

Getting Started

One of the easiest ways to compare two frameworks is by installing them and taking note how easily a user can get started with it, which is exactly what we will do next. We will try setting up Django and Flask on a Linux machine and create an app to see how easy (or difficult) the process is with each one.

Setting up Django

In this section, we will setup Django on an Linux-powered machine. Best way to get started with any Python framework is by using virtual environments. We will install it using pip.

$ sudo apt-get install python3-pip
$ pip3 install virtualenv
$ virtualenv --python=`which python3` ~/.virtualenvs/django_env

Note: If the pip3 command gives you an error, you may need to prefix it with sudo to make it work.

Once we're done setting up our virtual environment, which we've named django_env, we must activate it to start using it:

$ source ~/.virtualenvs/django_env/bin/activate

Once activated, we can finally install Django:

$ pip install Django

Suppose our project is called mysite. Make a new directory and enter it, run following commands:

$ mkdir mysite
$ cd mysite
$ django-admin startproject mysite

If you inspect the resulting project, your directory structure will be shown as:

mysite/  
  manage.py 
  mysite/
    __init__.py 
    settings.py 
    urls.py 
    wsgi.py

Let's take a look at what is significant about each of the directories and files that were created.

From here, we can actually run the app using the manage.py tool. The following command does some system checks, checks for database migrations, and some other things before actually running your server:

$ python manage.py runserver

Performing system checks...

System check identified no issues (0 silenced).

You have unapplied migrations; your app may not work properly until they are applied.  
Run 'python manage.py migrate' to apply them.

September 20, 2017 - 15:50:53  
Django version 1.11, using settings 'mysite.settings'  
Starting development server at http://127.0.0.1:8000/  
Quit the server with CONTROL-C.  

Note: Running your server in this way is meant for development only, and not production environments.

To check out your app, head to http://localhost:8000/, where you should see a page saying "It worked!".

But wait, you're still not done! To actually create any pages/functionality in your site, you need to create an app within your project. But why do you need an app? In Django, apps are web applications that do something, which could be a blog, a forum, or a commenting system. The project is a collection of your apps, as well as configuration for the apps and entire website.

So, to create your app, move in to your project directory and run the following command:

$ cd mysite
$ python manage.py startapp myapp

This will create another directory structure where you can actually manage your models, views, etc.

manage.py  
myapp/  
  __init__.py
  admin.py
  apps.py
  migrations/
  models.py
  tests.py
  views.py
mysite/  
  __init__.py 
  settings.py 
  urls.py 
  wsgi.py

From here, you need to set up your views in views.py and URL routing in urls.py, which we'll save for another tutorial.

But you get the point, right? It takes a few commands and quite a few files to get your Django project up and running.

Setting up Flask

Just like Django, we will use a virtual environment with Flask as well. So the commands for activating a virtual environment will remain the same as before. After that, instead of installing Django, we'll install Flask instead.

$ pip install Flask

Once the installation completes, we can start creating our Flask application. Now, unlike Django, Flask doesn't have a complicated directory structure. The structure of your Flask project is entirely up to you.

Borrowing an example from the Flask homepage, you can create a runnable Flask app from just a single file:

from flask import Flask  
app = Flask(__name__)

@app.route("/")
def hello():  
    return "Hello World!"

And running the app is about as easy as setting it up:

$ FLASK_APP=hello.py flask run
 * Running on http://localhost:5000/

Visiting the URL http://localhost:5000/ should display the text "Hello World!" in your browser.

I'd encourage you to look for some sample apps on the Flask homepage to learn more. Learning by example is one of the best ways to get up and running quickly.

The framework that "wins" this area is really up to your needs and experience. Django may be more favorable for beginners since it makes decisions for you (i.e. how to structure your app), whereas in Flask you need to handle this yourself.

On the other hand, Flask is simpler to get running since it requires very little to get going. An entire Flask app can be composed from a single file. The trade-offs really depend no what you need most.

Learning Curve

Regarding learning curve, as we saw in the last section with Flask, it was very easy to get started. The app doesn't require a complicated directory structure where you needed to remember which directory/file does what. Instead, you can add files and directories as you go according to your usage. This is what Flask is about, as a micro-framework for web development.

Django, on the other hand, has a bit higher learning curve since it is more "picky" about how things are set up and work. Because of this, you need to take more time learning how to compose modules and work within the confines of the framework.

This isn't all bad though, since this allows you to easily plug in 3rd party components in to your app without having to do any work integrating them.

Employability

Which of these frameworks will help you land a job? For many developers, this is one of the more important question regarding certain libraries and frameworks: which will help me get hired?

Django has quite a few large companies on its resume, which is because many companies that use Python for web development tend to use (or at least started off with) Django to power their site. Django, being a full-fledged framework, is often used early on in development because you get much more resources and power with it out of the box.

Here are just a few companies that use (or have used) Django for their sites:

Flask is a bit harder to gauge here, mostly because of the way it is used. Flask tends to be used more for microservices, which makes it harder to tell which companies are using it. Plus, companies with a microservice architecture are less likely to say their service is "powered by Flask" since they likely have many services potentially using many different frameworks.

There are, however, hints out there of who uses Flask based on job postings, tech talks, blog posts, etc. From those, we know that the following companies have used Flask somewhere in their backend infrastructure:

While Django may be more popular among companies, Flask is arguably more common among the more tech-focused companies as they are more likely to use microservices, and therefore micro-frameworks like Flask.

Project Size and Scope

Our comparison of each framework can become very subjective thanks to many different factors, like project scope, developer experience, type of project, etc. If the project is fairly small and it doesn't need all of the overhead that Django comes with, then Flask is the ideal choice to get started and get something done very quickly.

However, if the project is larger in duration and scope, then Django is likely the way to go as it already includes much of what you need. This basically means that many common components of a web-service/website either already comes with Django, or it is already available through 3rd party open source software. In some cases you can just create a Django project, plug in a bunch of components, create your views/templates, and you're done.

While we do praise Django for its extensibility, we can't ignore that Flask does have some extensions of its own. While they aren't quite as big in scope as Django (and many of these extensions come standard in Django), it's a step in the right direction.

Django's add-on components can be as big as a blog add-on, to as small as small middleware input validation. Most of Flask's extensions are small middleware components, which is still better than nothing and very helpful, considering the average size of Flask projects.

Limitations

Every piece of tech has its problems, and these frameworks are no different. So before you choose which to use, you might want to know what disadvantages each has, which we'll be talking about in this section.

Django

So, what are the aspects of Django that work against it to be selected as your framework of choice?

Django is a very large project. Once a developer, especially beginners, starts learning Django, it's easy for them to get lost in the source code, the built-in features, and components it provides, without even using them in an app.

Django is a fairly large framework to deploy for simple use-cases, as it hides much of the control from you. If you want to use something that isn't "standard" in Django, then you have to put in some extra work to do so.

Understanding components in Django can be a little difficult and tricky at times and can lead to tough decisions, like deciding if an existing component will work for your use-case, or if it'll end up causing you more work than it is worth.

Flask

Now that we've seen some of the problems with Django, let's not forget about Flask. Since the Flask framework is so small, there isn't a lot to complain about. Well, except for that fact right there: It's so small.

Flask is a micro-framework, which means it only provides the bare-bones functionality to get you started. This doesn't mean it can't be powerful and can't scale, it just means that you'll have to create much of the functionality of your service yourself. This means you'll need to handle integrating your database, data validation, file serving, etc.

While this could be considered an advantage to those who want control over everything, it also means it'll take you longer to get set up with a fully-functional website.

Choosing Flask or Django

While it's easy to talk about what each framework does and doesn't do, let's try and make a more direct comparison of each, which we'll do in this section.

When simplicity is a factor, Flask is the way to go. It allows much more control over your app and lets you decide how you want to implement things in a project. In contrast to this, Django provides a more inclusive experience, such as providing a default admin panel for your data, an ORM on top of your database, and protection against things like SQL injection, cross-site scripting, CSRF, etc.

If you put a lot of emphasis on community support, then Django is probably better in this regard given its history. It has been around since 2005, whereas Flask was created in 2010. At the time of writing this article, Django has about 3.5x more questions/answers on Stack Overflow than Flask (about 2600 Django questions to Flask's 750).

The Flask framework is relatively lightweight. In fact, it's almost 2.5x smaller than Django in terms of amount of code. That's a big difference, especially if you need to understand the inner-workings of your web framework. In this aspect, Flask will be much easier to read and understand for most developers.

Flask should be selected for development if you need complete control over your app, which ORM you want to use, which database you need to integrate with excellent opportunities to learn more about web services. Django, on the other hand, is better when there is a more clear path to creating what you want, or you're creating something that has been done before. For example, a blog would be a good use-case for Django.

Learn More

Want to learn more about either of these frameworks? There are quite a few resources out there. Here are a few courses that I've found to be pretty helpful, and will get you up to speed much quicker:

Flask vs Django
Python and Django Full Stack Web Developer Bootcamp

Flask vs Django
REST APIs with Flask and Python

Otherwise you can also get a great start by visiting each framework's respective websites:

Either way, the most important thing is you actually try them out, work through some examples, and decide on your own which is best for you.

Conclusion

In this article, we compared the two web frameworks, Django and Flask, by looking at their different properties and setting up a simple "Hello World!" app with each one.

You may find that if you're new to web development and decide to learn Django, it may take you a bit longer to truly understand what all of the underlying components do, and how to change them to actually do what you want. But there are many positives as well, and once you become proficient with Django, it'll end up saving you a lot of time in the end, given its huge list of components and vast community support.

A more advanced comparison for any frameworks can be done only with advanced use cases and scenarios. Just know that you can't really go wrong with either one, and learning either will set you up well for finding a job.

If you need a recommendation, then I'd personally go with Flask. By learning a framework that doesn't hide so much stuff from you, you can learn much, much more. Once you have a better understanding of the core concepts of web development and HTTP, you can start to use add-ons that abstract this away from you. But having that solid foundation of understanding is more important early on, in my opinion.

Which framework do you use, and why? Let us know in the comments!

21 Sep 2017 3:52pm GMT

Continuum Analytics Blog: Anaconda to Present at Strata Data Conference, New York

Anaconda, the most popular Python data science platform provider, today announced that several company experts will present two sessions and one tutorial at The Strata Data Conference on September 26 and 27 at the Javits Center in New York City.

21 Sep 2017 2:11pm GMT

Continuum Analytics Blog: Anaconda to Present at Strata Data Conference, New York

Anaconda, the most popular Python data science platform provider, today announced that several company experts will present two sessions and one tutorial at The Strata Data Conference on September 26 and 27 at the Javits Center in New York City.

21 Sep 2017 2:11pm GMT

DataCamp: How Not To Plot Hurricane Predictions

Visualizations help us make sense of the world and allow us to convey large amounts of complex information, data and predictions in a concise form. Expert predictions that need to be conveyed to non-expert audiences, whether they be the path of a hurricane or the outcome of an election, always contain a degree of uncertainty. If this uncertainty is not conveyed in the relevant visualizations, the results can be misleading and even dangerous.

Here, we explore the role of data visualization in plotting the predicted paths of hurricanes. We explore different visual methods to convey the uncertainty of expert predictions and the impact on layperson interpretation. We connect this to a broader discussion of best practices with respect to how news media outlets report on both expert models and scientific results on topics important to the population at large.

No Spaghetti Plots?

We have recently seen the damage wreaked by tropical storm systems in the Americas. News outlets such as the New York Times have conveyed a great deal of what has been going on using interactive visualizations for Hurricanes Harvey and Irma, for example. Visualizations include geographical visualisation of percentage of people without electricity, amount of rainfall, amount of damage and number of people in shelters, among many other things.

One particular type of plot has understandably been coming up recently and raising controversy: how to plot the predicted path of a hurricane, say, over the next 72 hours. There are several ways to visualize predicted paths, each way with its own pitfalls and misconceptions. Recently, we even saw an article in Ars Technica called Please, please stop sharing spaghetti plots of hurricane models, directed at Nate Silver and fivethirtyeight.

In what follows, I'll compare three common ways, explore their pros and cons and make suggestions for further types of plots. I'll also delve into why these types are important, which will help us decide which visual methods and techniques are most appropriate.

Disclaimer: I am definitively a non-expert in metereological matters and hurricane forecasting. But I have thought a lot about visual methods to convey data, predictions and models. I welcome and actively encourage the feedback of experts, along with that of others.

Visualizing Predicted Hurricane Paths

There are three common ways of creating visualizations for predicted hurricane paths. Before talking about at them, I want you to look at them and consider what information you can get from each of them. Do your best to interpret what each of them is trying to tell you, in turn, and then we'll delve into what their intentions are, along with their pros and cons:

The Cone of Uncertainty

spaghetti plot hurricane

From the National Hurricane Center

Spaghetti Plots (Type I)

spaghetti plot python

From South Florida Water Management District via fivethirtyeight

Spaghetti Plots (Type II)

From The New York Times. Surrounding text tells us 'One of the best hurricane forecasting systems is a model developed by an independent intergovernmental organization in Europe, according to Jeff Masters, a founder of the Weather Underground. The system produces 52 distinct forecasts of the storm's path, each represented by a line [above].'

Interpretation and Impact of Visualizations of Hurricanes' Predicted Paths

The Cone of Uncertainty

The cone of uncertainty, a tool used by the National Hurricane Center (NHC) and communicated by many news outlets, shows us the most likely path of the hurricane over the next five days, given by the black dots in the cone. It also shows how certain they are of this path. As time goes on, the prediction is less certain and this is captured by the cone, in that there is an approximately 66.6% chance that the centre of the hurricane will fall in the bounds of the cone.

Was this apparent from the plot itself?

It wasn't to me initially and I gathered this information from the plot itself, the NHC's 'about the cone of uncertainty' page and weather.com's demystification of the cone post. There are three more salient points, all of which we'll return to:

  • It is a common initial misconception that the widening of the cone over time suggests that the storm will grow;
  • The plot contains no information about the size of the storm, only about the potential path of its centre, and so is of limited use in telling us where to expect, for example, hurricane-force winds;
  • There is essential information contained in the text that accompanies the visualization, as well as the visualization itself, such as the note placed prominently at the top, '[t]he cone contains the probable path of the storm center but does not show the size of the storm...'; when judging the efficacy of a data visualization, we'll need to take into consideration all its properties, including text (and whether we can actually expect people to read it!); note that interactivity is a property that these visualizations do not have (but maybe should).

Spaghetti Plots (Type I)

This type of plots shows several predictions in one plot. One any given Type I spaghetti plot, the visualized trajectories are predictions from models from different agencies (NHC, the National Oceanic and Atmospheric Administration and the UK Met Office, for example). They are useful in that, like the cone of uncertainty, they inform us of the general region that may be in the hurricane's path. They are wonderfully unuseful and actually misleading in the fact that they weight each model (or prediction) equally.

In the Type I spaghetti plot above, there are predictions with varying degree uncertainty from agencies that have previously made predictions with variable degrees of success. So some paths are more likely than others, given what we currently know. This information is not present. Even more alarmingly, some of the paths are barely even predictions. Take the black dotted line XTRP, which is a straight-line prediction given the storm's current trajectory. This is not even a model. Eric Berger goes into more detail in this Ars Technica article.

Essentially, this type of plots provide an ensemble model (compare with aggregate polling). Yet, a key aspect of ensemble models is that each model is given an appropriate weight and these weights need be communicated in any data visualization. We'll soon see how to do this using a variation on Type I.

Spaghetti Plots (Type II)

These plots show many, say 50, different realizations of any given model. The point is that if we simulate (run) a model several times, it will given a different trajectory each time. Why? Nate Cohen put it well in The Upshot:

"It's really tough to forecast exactly when a storm will make a turn. Even a 15- or 20-mile difference in when it turns north could change whether Miami is hit by the eye wall, the fierce ring of thunderstorms that include the storm's strongest winds and surround the calmer eye."

These are perhaps my favourite of the three for several reasons:

  • By simulating multiple runs of the model, they provide an indication of the uncertainty underlying each model;
  • They give a picture of relative likelihood of the storm centre going through any given location. Put simply, if more of the plotted trajectories go through location A than through location B, then under the current model it is more likely that the centre of the storm will go through location A;
  • They are unlikely to be misinterpreted (at least compared to the cone of uncertainty and the Type I plots). All the words required on the visualization are 'Each line represents one forecast of Irma's path'.

One con of Type II is that they are not representative of multiple models but, as we'll see, this can be altered by combining them with Type I plots. Another con is that they, like the others, only communicate the path of the centre of the storm and say nothing about its size. Soon we'll also see how we can remedy this. Note that the distinction between Type I and Type II spaghetti plots is not one that I have found in the literature, but one that I created because these plots have such different interpretations and effects.

For the time being, however, note that we've been discussing the efficacy of certain types of plots without explicitly discussing their purpose, that is, why we need them at all. Before going any further, let's step back a bit and try to answer the question 'What is the purpose of visualizing the predicted path of a hurricane?' Performing such ostensibly naive tasks is often illuminating.

Why Plot Predicted Paths of Hurricanes?

Why are we trying to convey the predicted path of a tropical storm? I'll provide several answers to this in a minute.

But first, let me say what these visualizations are not intended for. We are not using these visualizations to help people decide whether or not to evacuate their homes or towns. Ordering or advising evacuation is something that is done by local authorities, after repeated consultation with experts, scientists, modelers and other key stakeholders.

The major point of this type of visualization is to allow the general populace to be as well-informed as possible about the possible paths of the hurricane and allow them to prepare for the worst if there's a chance that where they are or will be is in the path of destruction. It is not to unduly scare people. As weather.com states with respect to the function of the cone of uncertainty, '[e]ach tropical system is given a forecast cone to help the public better understand where it's headed' and '[t]he cone is designed to show increasing forecast uncertainty over time.'

To this end, I think that an important property would be for a reader to be able to look at it and say 'it is very likely/likely/50% possible/not likely/very unlikely' that my house (for example) will be significantly damaged by the hurricane.

Even better, to be able to say "There's a 30-40% chance, given the current state-of-the-art modeling, that my house will be significantly damaged".

Then we have a hierarchy of what we want our visualization to communicate:

  • At a bare minimum, we want civilians to be aware of the possible paths of the hurricane.
  • Then we would like civilians to be able to say whether it is very likely, likely, unlikely or very unlikely that their house, for example, is in the path.
  • Ideally, a civilian would look at the visualization and be able to read off quantitatively what the probability (or range of probabilities) of their house being in the hurricane's path is.

On top of this, we want our visualizations to be neither misleading nor easy to misinterpret.

The Cone of Uncertainty versus Spaghetti Plots

All three methods perform the minimum required function, to alert civilians to the possible paths of the hurricane. The cone of uncertainty does a pretty good job at allowing a civilian to say how likely it is that a hurricane goes through a particular location (within the cone, it's about two-thirds likely). At least qualitatively, Type II spaghetti plots also do a good job here, as described above, 'if more of the trajectories go through location A than through location B, then under the current model it is more likely that the centre of the storm will go through location A'.

If you plot 50 trajectories, you get a sense of where the centre of the storm will likely be, that is, if around half of the trajectories go through a location, then there's an approximately 50% chance (according to our model) that the centre of the storm will hit that location. None of these methods yet perform the 3rd function and we'll see below how combining Type I and Type II spaghetti plots will allow us to do this.

The major problem with the cone of uncertainty and Type I spaghetti models is that the cone of uncertainty is easy to misinterpret (in that many people interpret the cone as a growing storm and do not appreciate the role of uncertainty) and that the Type I spaghetti models are misleading (they make all models look equally believable). These models then don't satisfy the basic requirement that 'we want our visualizations to be neither misleading nor easy to misinterpret.'

Best Practices for Visualizing Hurricane Prediction Paths

Type II spaghetti plots are the most descriptive and the least open to misinterpretation. But they do fail at presenting the results of all models. That is, they don't aggregate over multiple models like we saw in Type I.

So what if we combined Type I and Type II?

To answer this, I did a small experiment using python, folium and numpy. You can find all the code here.

I first took one the NHC's Hurricane Irma's prediction paths from last week, added some random noise and plotted 50 trajectories. Note that, once again, I am a non-expert in all matters meteorological. The noise that I generated and added to the predicted signal/path was not based on any models and, in a real use case, would come from the models themselves (if you're interested, I used Gaussian noise). For the record, I also found it difficult to find data concerning any of the predicted paths reported in the media. The data I finally used I found here.

Here's a simple Type II spaghetti plot with 50 trajectories:

But these are possible trajectories generated by a single model. What if we had multiple models from different agencies? Well, we can plot 50 trajectories from each:

One of the really cool aspects of Type II spaghetti plots is that, if we plot enough of them, each trajectory becomes indistinct and we begin to see a heatmap of where the centre of the hurricane is likely to be. All this means is that the more blue in a given region, the more likely it is for the path to go through there. Zoom in to check it out.

Moreover, if we believe that one model is more likely than another (if, for example, the experts who produced that model have produced far more accurate models previously), we can weight these models accordingly via, for example, transparency of the trajectories, as we do below. Note that weighting these models is a task for an expert and an essential part of this process of aggregate modeling.

What the above does is solve the tasks required by the first two properties that we want our visualizations to have. To achieve the 3rd, a reader being able to read off that it's, say 30-40% likely for the centre of a hurricane to pass through a particular location, there are two solutions:

  • to alter the heatmap so that it moves between, say, red and blue and include a key that says, for example, red means a probability of greater than 90%;
  • To transform the heatmap into a contour map that shows regions in which the probability takes on certain values.

Also, do note that this will tell somebody the probability that a given location will be hit by the hurricane's center. You could combine (well, convolve) this with information about the size of the hurricane to transform the heatmap into one of the probability of a location being hit by hurricane-force winds. If you'd like to do this, go and hack around the code that I wrote to generate the plots above (I plan to write a follow-up post doing this and walking through the code).

Visualizing Uncertainty and Data Journalism

What can we take away from this? We have explored several types of visualization methods for predicted hurricane paths, discussed the pros and cons of each and suggested a way forward for more informative and less misleading plots of such paths, plots that communicate not only the results but also the uncertainty around the models.

This is part of a broader conversation that we need to be having about reporting uncertainty in visualizations and data journalism, in general. We need to actively participate in conversations about how experts report uncertainty to civilians via news media outlets. Here's a great piece from The Upshot demonstrating what the jobs report could look like due to statistical noise, even if jobs were steady. Here's another Upshot piece showing the role of noise and uncertainty in interpreting polls. I'm well aware that we need headlines to sell news and the role of click-bait in the modern news media landscape, but we need to be communicating not merely results, but uncertainty around those results so as not mislead the general public and potentially ourselves. Perhaps more importantly, the education system needs to shift and equip all civilians with levels of data literacy and statistical literacy in order to deal with this movement into the data-driven age. We can all contribute to this.

21 Sep 2017 12:59pm GMT

DataCamp: How Not To Plot Hurricane Predictions

Visualizations help us make sense of the world and allow us to convey large amounts of complex information, data and predictions in a concise form. Expert predictions that need to be conveyed to non-expert audiences, whether they be the path of a hurricane or the outcome of an election, always contain a degree of uncertainty. If this uncertainty is not conveyed in the relevant visualizations, the results can be misleading and even dangerous.

Here, we explore the role of data visualization in plotting the predicted paths of hurricanes. We explore different visual methods to convey the uncertainty of expert predictions and the impact on layperson interpretation. We connect this to a broader discussion of best practices with respect to how news media outlets report on both expert models and scientific results on topics important to the population at large.

No Spaghetti Plots?

We have recently seen the damage wreaked by tropical storm systems in the Americas. News outlets such as the New York Times have conveyed a great deal of what has been going on using interactive visualizations for Hurricanes Harvey and Irma, for example. Visualizations include geographical visualisation of percentage of people without electricity, amount of rainfall, amount of damage and number of people in shelters, among many other things.

One particular type of plot has understandably been coming up recently and raising controversy: how to plot the predicted path of a hurricane, say, over the next 72 hours. There are several ways to visualize predicted paths, each way with its own pitfalls and misconceptions. Recently, we even saw an article in Ars Technica called Please, please stop sharing spaghetti plots of hurricane models, directed at Nate Silver and fivethirtyeight.

In what follows, I'll compare three common ways, explore their pros and cons and make suggestions for further types of plots. I'll also delve into why these types are important, which will help us decide which visual methods and techniques are most appropriate.

Disclaimer: I am definitively a non-expert in metereological matters and hurricane forecasting. But I have thought a lot about visual methods to convey data, predictions and models. I welcome and actively encourage the feedback of experts, along with that of others.

Visualizing Predicted Hurricane Paths

There are three common ways of creating visualizations for predicted hurricane paths. Before talking about at them, I want you to look at them and consider what information you can get from each of them. Do your best to interpret what each of them is trying to tell you, in turn, and then we'll delve into what their intentions are, along with their pros and cons:

The Cone of Uncertainty

spaghetti plot hurricane

From the National Hurricane Center

Spaghetti Plots (Type I)

spaghetti plot python

From South Florida Water Management District via fivethirtyeight

Spaghetti Plots (Type II)

From The New York Times. Surrounding text tells us 'One of the best hurricane forecasting systems is a model developed by an independent intergovernmental organization in Europe, according to Jeff Masters, a founder of the Weather Underground. The system produces 52 distinct forecasts of the storm's path, each represented by a line [above].'

Interpretation and Impact of Visualizations of Hurricanes' Predicted Paths

The Cone of Uncertainty

The cone of uncertainty, a tool used by the National Hurricane Center (NHC) and communicated by many news outlets, shows us the most likely path of the hurricane over the next five days, given by the black dots in the cone. It also shows how certain they are of this path. As time goes on, the prediction is less certain and this is captured by the cone, in that there is an approximately 66.6% chance that the centre of the hurricane will fall in the bounds of the cone.

Was this apparent from the plot itself?

It wasn't to me initially and I gathered this information from the plot itself, the NHC's 'about the cone of uncertainty' page and weather.com's demystification of the cone post. There are three more salient points, all of which we'll return to:

  • It is a common initial misconception that the widening of the cone over time suggests that the storm will grow;
  • The plot contains no information about the size of the storm, only about the potential path of its centre, and so is of limited use in telling us where to expect, for example, hurricane-force winds;
  • There is essential information contained in the text that accompanies the visualization, as well as the visualization itself, such as the note placed prominently at the top, '[t]he cone contains the probable path of the storm center but does not show the size of the storm...'; when judging the efficacy of a data visualization, we'll need to take into consideration all its properties, including text (and whether we can actually expect people to read it!); note that interactivity is a property that these visualizations do not have (but maybe should).

Spaghetti Plots (Type I)

This type of plots shows several predictions in one plot. One any given Type I spaghetti plot, the visualized trajectories are predictions from models from different agencies (NHC, the National Oceanic and Atmospheric Administration and the UK Met Office, for example). They are useful in that, like the cone of uncertainty, they inform us of the general region that may be in the hurricane's path. They are wonderfully unuseful and actually misleading in the fact that they weight each model (or prediction) equally.

In the Type I spaghetti plot above, there are predictions with varying degree uncertainty from agencies that have previously made predictions with variable degrees of success. So some paths are more likely than others, given what we currently know. This information is not present. Even more alarmingly, some of the paths are barely even predictions. Take the black dotted line XTRP, which is a straight-line prediction given the storm's current trajectory. This is not even a model. Eric Berger goes into more detail in this Ars Technica article.

Essentially, this type of plots provide an ensemble model (compare with aggregate polling). Yet, a key aspect of ensemble models is that each model is given an appropriate weight and these weights need be communicated in any data visualization. We'll soon see how to do this using a variation on Type I.

Spaghetti Plots (Type II)

These plots show many, say 50, different realizations of any given model. The point is that if we simulate (run) a model several times, it will given a different trajectory each time. Why? Nate Cohen put it well in The Upshot:

"It's really tough to forecast exactly when a storm will make a turn. Even a 15- or 20-mile difference in when it turns north could change whether Miami is hit by the eye wall, the fierce ring of thunderstorms that include the storm's strongest winds and surround the calmer eye."

These are perhaps my favourite of the three for several reasons:

  • By simulating multiple runs of the model, they provide an indication of the uncertainty underlying each model;
  • They give a picture of relative likelihood of the storm centre going through any given location. Put simply, if more of the plotted trajectories go through location A than through location B, then under the current model it is more likely that the centre of the storm will go through location A;
  • They are unlikely to be misinterpreted (at least compared to the cone of uncertainty and the Type I plots). All the words required on the visualization are 'Each line represents one forecast of Irma's path'.

One con of Type II is that they are not representative of multiple models but, as we'll see, this can be altered by combining them with Type I plots. Another con is that they, like the others, only communicate the path of the centre of the storm and say nothing about its size. Soon we'll also see how we can remedy this. Note that the distinction between Type I and Type II spaghetti plots is not one that I have found in the literature, but one that I created because these plots have such different interpretations and effects.

For the time being, however, note that we've been discussing the efficacy of certain types of plots without explicitly discussing their purpose, that is, why we need them at all. Before going any further, let's step back a bit and try to answer the question 'What is the purpose of visualizing the predicted path of a hurricane?' Performing such ostensibly naive tasks is often illuminating.

Why Plot Predicted Paths of Hurricanes?

Why are we trying to convey the predicted path of a tropical storm? I'll provide several answers to this in a minute.

But first, let me say what these visualizations are not intended for. We are not using these visualizations to help people decide whether or not to evacuate their homes or towns. Ordering or advising evacuation is something that is done by local authorities, after repeated consultation with experts, scientists, modelers and other key stakeholders.

The major point of this type of visualization is to allow the general populace to be as well-informed as possible about the possible paths of the hurricane and allow them to prepare for the worst if there's a chance that where they are or will be is in the path of destruction. It is not to unduly scare people. As weather.com states with respect to the function of the cone of uncertainty, '[e]ach tropical system is given a forecast cone to help the public better understand where it's headed' and '[t]he cone is designed to show increasing forecast uncertainty over time.'

To this end, I think that an important property would be for a reader to be able to look at it and say 'it is very likely/likely/50% possible/not likely/very unlikely' that my house (for example) will be significantly damaged by the hurricane.

Even better, to be able to say "There's a 30-40% chance, given the current state-of-the-art modeling, that my house will be significantly damaged".

Then we have a hierarchy of what we want our visualization to communicate:

  • At a bare minimum, we want civilians to be aware of the possible paths of the hurricane.
  • Then we would like civilians to be able to say whether it is very likely, likely, unlikely or very unlikely that their house, for example, is in the path.
  • Ideally, a civilian would look at the visualization and be able to read off quantitatively what the probability (or range of probabilities) of their house being in the hurricane's path is.

On top of this, we want our visualizations to be neither misleading nor easy to misinterpret.

The Cone of Uncertainty versus Spaghetti Plots

All three methods perform the minimum required function, to alert civilians to the possible paths of the hurricane. The cone of uncertainty does a pretty good job at allowing a civilian to say how likely it is that a hurricane goes through a particular location (within the cone, it's about two-thirds likely). At least qualitatively, Type II spaghetti plots also do a good job here, as described above, 'if more of the trajectories go through location A than through location B, then under the current model it is more likely that the centre of the storm will go through location A'.

If you plot 50 trajectories, you get a sense of where the centre of the storm will likely be, that is, if around half of the trajectories go through a location, then there's an approximately 50% chance (according to our model) that the centre of the storm will hit that location. None of these methods yet perform the 3rd function and we'll see below how combining Type I and Type II spaghetti plots will allow us to do this.

The major problem with the cone of uncertainty and Type I spaghetti models is that the cone of uncertainty is easy to misinterpret (in that many people interpret the cone as a growing storm and do not appreciate the role of uncertainty) and that the Type I spaghetti models are misleading (they make all models look equally believable). These models then don't satisfy the basic requirement that 'we want our visualizations to be neither misleading nor easy to misinterpret.'

Best Practices for Visualizing Hurricane Prediction Paths

Type II spaghetti plots are the most descriptive and the least open to misinterpretation. But they do fail at presenting the results of all models. That is, they don't aggregate over multiple models like we saw in Type I.

So what if we combined Type I and Type II?

To answer this, I did a small experiment using python, folium and numpy. You can find all the code here.

I first took one the NHC's Hurricane Irma's prediction paths from last week, added some random noise and plotted 50 trajectories. Note that, once again, I am a non-expert in all matters meteorological. The noise that I generated and added to the predicted signal/path was not based on any models and, in a real use case, would come from the models themselves (if you're interested, I used Gaussian noise). For the record, I also found it difficult to find data concerning any of the predicted paths reported in the media. The data I finally used I found here.

Here's a simple Type II spaghetti plot with 50 trajectories:

But these are possible trajectories generated by a single model. What if we had multiple models from different agencies? Well, we can plot 50 trajectories from each:

One of the really cool aspects of Type II spaghetti plots is that, if we plot enough of them, each trajectory becomes indistinct and we begin to see a heatmap of where the centre of the hurricane is likely to be. All this means is that the more blue in a given region, the more likely it is for the path to go through there. Zoom in to check it out.

Moreover, if we believe that one model is more likely than another (if, for example, the experts who produced that model have produced far more accurate models previously), we can weight these models accordingly via, for example, transparency of the trajectories, as we do below. Note that weighting these models is a task for an expert and an essential part of this process of aggregate modeling.

What the above does is solve the tasks required by the first two properties that we want our visualizations to have. To achieve the 3rd, a reader being able to read off that it's, say 30-40% likely for the centre of a hurricane to pass through a particular location, there are two solutions:

  • to alter the heatmap so that it moves between, say, red and blue and include a key that says, for example, red means a probability of greater than 90%;
  • To transform the heatmap into a contour map that shows regions in which the probability takes on certain values.

Also, do note that this will tell somebody the probability that a given location will be hit by the hurricane's center. You could combine (well, convolve) this with information about the size of the hurricane to transform the heatmap into one of the probability of a location being hit by hurricane-force winds. If you'd like to do this, go and hack around the code that I wrote to generate the plots above (I plan to write a follow-up post doing this and walking through the code).

Visualizing Uncertainty and Data Journalism

What can we take away from this? We have explored several types of visualization methods for predicted hurricane paths, discussed the pros and cons of each and suggested a way forward for more informative and less misleading plots of such paths, plots that communicate not only the results but also the uncertainty around the models.

This is part of a broader conversation that we need to be having about reporting uncertainty in visualizations and data journalism, in general. We need to actively participate in conversations about how experts report uncertainty to civilians via news media outlets. Here's a great piece from The Upshot demonstrating what the jobs report could look like due to statistical noise, even if jobs were steady. Here's another Upshot piece showing the role of noise and uncertainty in interpreting polls. I'm well aware that we need headlines to sell news and the role of click-bait in the modern news media landscape, but we need to be communicating not merely results, but uncertainty around those results so as not mislead the general public and potentially ourselves. Perhaps more importantly, the education system needs to shift and equip all civilians with levels of data literacy and statistical literacy in order to deal with this movement into the data-driven age. We can all contribute to this.

21 Sep 2017 12:59pm GMT

PyCharm: PyCharm 2017.3 EAP 2

After a strong start, we continue our early access program (EAP) with its second release. Download EAP 2 now!

Testing RESTful Applications

Many of us work on web applications which expose a RESTful API, or at least an API that pretends to be RESTful. To test these some of us use cURL, some browser extension, or some other piece of software. There is a REST client in PyCharm, but we've decided it can use some improvement, so we're making an all new one.

The new REST client is entirely editor based, you write your request in a file, and then run the request to get a response. Sounds easy enough, right?

To see how it works, we'll take the sample application from Flask-Restplus, which as you might expect exposes a todo API.

We'll start out by creating a new todo. This is done by POST-ing to the /todos/ endpoint. To use the new PyCharm REST client, we should start by creating a .http file. If we don't intend to save this, we can create a scratch file. Press Ctrl+Alt+Shift+Insert (Shift+Command+N on macOS) to start creating a scratch file and choose 'HTTP request' as the type. Let's type our request into the file:

### Post a todo

POST http://localhost:5000/todos/
Accept: application/json
Content-Type: application/json

{
    "task": "Create my task!"
}

Now click the green play button next to the first line, and you should see that the task was created:

Create a Task with the REST client

You can see the response in the Run tool window, and you might also notice that PyCharm wrote a new line in our file with the name of a .json file. This file contains the response, so if we Ctrl+Click (Cmd+Click) the filename, or use Ctrl+B (Cmd+B) to go to definition we see the full response in a separate file.

Those files become really useful when we do the same request a couple times but get different results. If we use a GET request to get our todo, and then use a PUT to change it, and redo our GET, we'll now have two files there. We can then use the blue icon with the arrows to see the difference between the responses:

See the difference between REST responses

Now it's your turn! Get the EAP and try it yourself

Further Improvements

To read about all improvements in this version, see the release notes

As always, we really appreciate your feedback! Please let us know on YouTrack about any issues you experience or suggestions for improvement. You can also reach us on Twitter, or by leaving a comment below.

21 Sep 2017 11:44am GMT

PyCharm: PyCharm 2017.3 EAP 2

After a strong start, we continue our early access program (EAP) with its second release. Download EAP 2 now!

Testing RESTful Applications

Many of us work on web applications which expose a RESTful API, or at least an API that pretends to be RESTful. To test these some of us use cURL, some browser extension, or some other piece of software. There is a REST client in PyCharm, but we've decided it can use some improvement, so we're making an all new one.

The new REST client is entirely editor based, you write your request in a file, and then run the request to get a response. Sounds easy enough, right?

To see how it works, we'll take the sample application from Flask-Restplus, which as you might expect exposes a todo API.

We'll start out by creating a new todo. This is done by POST-ing to the /todos/ endpoint. To use the new PyCharm REST client, we should start by creating a .http file. If we don't intend to save this, we can create a scratch file. Press Ctrl+Alt+Shift+Insert (Shift+Command+N on macOS) to start creating a scratch file and choose 'HTTP request' as the type. Let's type our request into the file:

### Post a todo

POST http://localhost:5000/todos/
Accept: application/json
Content-Type: application/json

{
    "task": "Create my task!"
}

Now click the green play button next to the first line, and you should see that the task was created:

Create a Task with the REST client

You can see the response in the Run tool window, and you might also notice that PyCharm wrote a new line in our file with the name of a .json file. This file contains the response, so if we Ctrl+Click (Cmd+Click) the filename, or use Ctrl+B (Cmd+B) to go to definition we see the full response in a separate file.

Those files become really useful when we do the same request a couple times but get different results. If we use a GET request to get our todo, and then use a PUT to change it, and redo our GET, we'll now have two files there. We can then use the blue icon with the arrows to see the difference between the responses:

See the difference between REST responses

Now it's your turn! Get the EAP and try it yourself

Further Improvements

To read about all improvements in this version, see the release notes

As always, we really appreciate your feedback! Please let us know on YouTrack about any issues you experience or suggestions for improvement. You can also reach us on Twitter, or by leaving a comment below.

21 Sep 2017 11:44am GMT

10 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: King Willams Town Bahnhof

Gestern musste ich morgens zur Station nach KWT um unsere Rerservierten Bustickets für die Weihnachtsferien in Capetown abzuholen. Der Bahnhof selber ist seit Dezember aus kostengründen ohne Zugverbindung - aber Translux und co - die langdistanzbusse haben dort ihre Büros.


Größere Kartenansicht




© benste CC NC SA

10 Nov 2011 10:57am GMT

09 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein

Niemand ist besorgt um so was - mit dem Auto fährt man einfach durch, und in der City - nahe Gnobie- "ne das ist erst gefährlich wenn die Feuerwehr da ist" - 30min später auf dem Rückweg war die Feuerwehr da.




© benste CC NC SA

09 Nov 2011 8:25pm GMT

08 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Brai Party

Brai = Grillabend o.ä.

Die möchte gern Techniker beim Flicken ihrer SpeakOn / Klinke Stecker Verzweigungen...

Die Damen "Mamas" der Siedlung bei der offiziellen Eröffnungsrede

Auch wenn weniger Leute da waren als erwartet, Laute Musik und viele Leute ...

Und natürlich ein Feuer mit echtem Holz zum Grillen.

© benste CC NC SA

08 Nov 2011 2:30pm GMT

07 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Lumanyano Primary

One of our missions was bringing Katja's Linux Server back to her room. While doing that we saw her new decoration.

Björn, Simphiwe carried the PC to Katja's school


© benste CC NC SA

07 Nov 2011 2:00pm GMT

06 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nelisa Haircut

Today I went with Björn to Needs Camp to Visit Katja's guest family for a special Party. First of all we visited some friends of Nelisa - yeah the one I'm working with in Quigney - Katja's guest fathers sister - who did her a haircut.

African Women usually get their hair done by arranging extensions and not like Europeans just cutting some hair.

In between she looked like this...

And then she was done - looks amazing considering the amount of hair she had last week - doesn't it ?

© benste CC NC SA

06 Nov 2011 7:45pm GMT

05 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Mein Samstag

Irgendwie viel mir heute auf das ich meine Blogposts mal ein bischen umstrukturieren muss - wenn ich immer nur von neuen Plätzen berichte, dann müsste ich ja eine Rundreise machen. Hier also mal ein paar Sachen aus meinem heutigen Alltag.

Erst einmal vorweg, Samstag zählt zumindest für uns Voluntäre zu den freien Tagen.

Dieses Wochenende sind nur Rommel und ich auf der Farm - Katja und Björn sind ja mittlerweile in ihren Einsatzstellen, und meine Mitbewohner Kyle und Jonathan sind zu Hause in Grahamstown - sowie auch Sipho der in Dimbaza wohnt.
Robin, die Frau von Rommel ist in Woodie Cape - schon seit Donnerstag um da ein paar Sachen zur erledigen.
Naja wie dem auch sei heute morgen haben wir uns erstmal ein gemeinsames Weetbix/Müsli Frühstück gegönnt und haben uns dann auf den Weg nach East London gemacht. 2 Sachen waren auf der Checkliste Vodacom, Ethienne (Imobilienmakler) außerdem auf dem Rückweg die fehlenden Dinge nach NeedsCamp bringen.

Nachdem wir gerade auf der Dirtroad losgefahren sind mussten wir feststellen das wir die Sachen für Needscamp und Ethienne nicht eingepackt hatten aber die Pumpe für die Wasserversorgung im Auto hatten.

Also sind wir in EastLondon ersteinmal nach Farmerama - nein nicht das onlinespiel farmville - sondern einen Laden mit ganz vielen Sachen für eine Farm - in Berea einem nördlichen Stadteil gefahren.

In Farmerama haben wir uns dann beraten lassen für einen Schnellverschluss der uns das leben mit der Pumpe leichter machen soll und außerdem eine leichtere Pumpe zur Reperatur gebracht, damit es nicht immer so ein großer Aufwand ist, wenn mal wieder das Wasser ausgegangen ist.

Fego Caffé ist in der Hemmingways Mall, dort mussten wir und PIN und PUK einer unserer Datensimcards geben lassen, da bei der PIN Abfrage leider ein zahlendreher unterlaufen ist. Naja auf jeden Fall speichern die Shops in Südafrika so sensible Daten wie eine PUK - die im Prinzip zugang zu einem gesperrten Phone verschafft.

Im Cafe hat Rommel dann ein paar online Transaktionen mit dem 3G Modem durchgeführt, welches ja jetzt wieder funktionierte - und übrigens mittlerweile in Ubuntu meinem Linuxsystem perfekt klappt.

Nebenbei bin ich nach 8ta gegangen um dort etwas über deren neue Deals zu erfahren, da wir in einigen von Hilltops Centern Internet anbieten wollen. Das Bild zeigt die Abdeckung UMTS in NeedsCamp Katjas Ort. 8ta ist ein neuer Telefonanbieter von Telkom, nachdem Vodafone sich Telkoms anteile an Vodacom gekauft hat müssen die komplett neu aufbauen.
Wir haben uns dazu entschieden mal eine kostenlose Prepaidkarte zu testen zu organisieren, denn wer weis wie genau die Karte oben ist ... Bevor man einen noch so billigen Deal für 24 Monate signed sollte man wissen obs geht.

Danach gings nach Checkers in Vincent, gesucht wurden zwei Hotplates für WoodyCape - R 129.00 eine - also ca. 12€ für eine zweigeteilte Kochplatte.
Wie man sieht im Hintergrund gibts schon Weihnachtsdeko - Anfang November und das in Südafrika bei sonnig warmen min- 25°C

Mittagessen haben wir uns bei einem Pakistanischen Curry Imbiss gegönnt - sehr empfehlenswert !
Naja und nachdem wir dann vor ner Stunde oder so zurück gekommen sind habe ich noch den Kühlschrank geputzt den ich heute morgen zum defrosten einfach nach draußen gestellt hatte. Jetzt ist der auch mal wieder sauber und ohne 3m dicke Eisschicht...

Morgen ... ja darüber werde ich gesondert berichten ... aber vermutlich erst am Montag, denn dann bin ich nochmal wieder in Quigney(East London) und habe kostenloses Internet.

© benste CC NC SA

05 Nov 2011 4:33pm GMT

31 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Sterkspruit Computer Center

Sterkspruit is one of Hilltops Computer Centres in the far north of Eastern Cape. On the trip to J'burg we've used the opportunity to take a look at the centre.

Pupils in the big classroom


The Trainer


School in Countryside


Adult Class in the Afternoon


"Town"


© benste CC NC SA

31 Oct 2011 4:58pm GMT

Benedict Stein: Technical Issues

What are you doing in an internet cafe if your ADSL and Faxline has been discontinued before months end. Well my idea was sitting outside and eating some ice cream.
At least it's sunny and not as rainy as on the weekend.


© benste CC NC SA

31 Oct 2011 3:11pm GMT

30 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nellis Restaurant

For those who are traveling through Zastron - there is a very nice Restaurant which is serving delicious food at reasanable prices.
In addition they're selling home made juices jams and honey.




interior


home made specialities - the shop in the shop


the Bar


© benste CC NC SA

30 Oct 2011 4:47pm GMT

29 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: The way back from J'burg

Having the 10 - 12h trip from J'burg back to ELS I was able to take a lot of pcitures including these different roadsides

Plain Street


Orange River in its beginngings (near Lesotho)


Zastron Anglican Church


The Bridge in Between "Free State" and Eastern Cape next to Zastron


my new Background ;)


If you listen to GoogleMaps you'll end up traveling 50km of gravel road - as it was just renewed we didn't have that many problems and saved 1h compared to going the official way with all it's constructions sites




Freeway


getting dark


© benste CC NC SA

29 Oct 2011 4:23pm GMT

28 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Wie funktioniert eigentlich eine Baustelle ?

Klar einiges mag anders sein, vieles aber gleich - aber ein in Deutschland täglich übliches Bild einer Straßenbaustelle - wie läuft das eigentlich in Südafrika ?

Ersteinmal vorweg - NEIN keine Ureinwohner die mit den Händen graben - auch wenn hier mehr Manpower genutzt wird - sind sie fleißig mit Technologie am arbeiten.

Eine ganz normale "Bundesstraße"


und wie sie erweitert wird


gaaaanz viele LKWs


denn hier wird eine Seite über einen langen Abschnitt komplett gesperrt, so das eine Ampelschaltung mit hier 45 Minuten Wartezeit entsteht


Aber wenigstens scheinen die ihren Spaß zu haben ;) - Wie auch wir denn gücklicher Weise mussten wir nie länger als 10 min. warten.

© benste CC NC SA

28 Oct 2011 4:20pm GMT