19 Aug 2018
Planet Python
PyBites: PyBites Twitter Digest - Issue 27, 2018
One more day for PSF Fellowship Nominations! Get them in ASAP!
Do you know someone who has served the Python community thru extraordinary efforts? Nominate them to be a PSF Fello… https://t.co/cNzFpaaxhN
- Python Software (@ThePSF) August 18, 2018
Decorator library to configure function arguments
Submitted by @clamytoe.
- Patrick Porto (@PatrickPorto) August 09, 2018
Mockaroo! Who knew? Create your own data to use in Sketch!
Submitted by @bohemianjack.
"Create your own #data to use in Sketch (no code required)" by @rgesulfo https://t.co/UQIjnBv3u7 - generating some… https://t.co/dqrySH0DkR
- Pybites (@pybites) August 11, 2018
Very cool use case of OpenCV
New tutorial published!🚀 Today we're building "People Counter". Learn how to count the # of people entering & leavi… https://t.co/JQYXfNxrnu
- Adrian Rosebrock (@PyImageSearch) August 13, 2018
Nice to see Netflix doing stuff like this
RT @MichelleUfford: At Netflix we're reimagining what a Jupyter notebook can be, who can use it, & what you can do with it. Check out my Ne…
- Enthought Inc. (@enthought) August 18, 2018
Deep Learning basics by Sentdex!
Just posted an updated Deep Learning basics with Python, TensorFlow and Keras tutorial: https://t.co/Vr3nvT7CRo… https://t.co/AqLE0jMaRG
- Harrison Kinsley🐍 (@Sentdex) August 11, 2018
This is a great security step. Nice going PyPI!
RT @pypi: PyPI will no longer accept passwords that have been published in data breaches. For background you can take a look at https://t.…
- Ewa Jodlowska (@ewa_jodlowska) August 14, 2018
This is why the UX is so important.
"The average user doesn't give a damn what happens, as long as (1) it works and (2) it's fast." - Daniel J. Bernstein
- Programming Wisdom (@CodeWisdom) August 17, 2018
Altair version 2.2 released
Altair version 2.2 just released! This adds support to Vega-Lite 2.6, as well as offering a number of API improveme… https://t.co/5oSmcCA5Pq
- Jake VanderPlas (@jakevdp) August 14, 2018
ripgrep is faster than {grep, ag, git grep, ucg, pt, sift} https://t.co/gTUnxyTMw2
- Regular Expression (@RegexTip) August 16, 2018
Django v Wordpress
Django vs WordPress: Learn how to decide between these 2 very different #ContentManagement systems:… https://t.co/DP1QdmFxrP
- Caktus Group (@CaktusGroup) August 17, 2018
How to use Bootstrap 4 forms in Django
RT @pybites: How to Use Bootstrap 4 Forms With #Django https://t.co/iL9gMxy8D3 via @vitorfs
- Martin Uribe (@clamytoe) August 15, 2018
PyConAU is all sold out! Who's going next week?
That's it! We're sold out! There are still some spaces left at our development sprints for main conference attende… https://t.co/WqyipT9dq6
- PyCon Australia (@pyconau) August 17, 2018
Python text-to-speech!
Text to speech conversion using python using gtts https://t.co/zyeDeIyiwJ
- Python LibHunt (@PythonLibHunt) August 18, 2018
Nice! A web UI for pdb!
Looks useful.. python-web-pdb: Web-based remote UI for Python's PDB debugger https://t.co/J4WgKJ3EuI
- Ned Batchelder (@nedbat) August 07, 2018
>>> from pybites import Bob, Julian Keep Calm and Code in Python!
19 Aug 2018 10:44am GMT
PyBites: PyBites Twitter Digest - Issue 27, 2018
One more day for PSF Fellowship Nominations! Get them in ASAP!
Do you know someone who has served the Python community thru extraordinary efforts? Nominate them to be a PSF Fello… https://t.co/cNzFpaaxhN
- Python Software (@ThePSF) August 18, 2018
Decorator library to configure function arguments
Submitted by @clamytoe.
- Patrick Porto (@PatrickPorto) August 09, 2018
Mockaroo! Who knew? Create your own data to use in Sketch!
Submitted by @bohemianjack.
"Create your own #data to use in Sketch (no code required)" by @rgesulfo https://t.co/UQIjnBv3u7 - generating some… https://t.co/dqrySH0DkR
- Pybites (@pybites) August 11, 2018
Very cool use case of OpenCV
New tutorial published!🚀 Today we're building "People Counter". Learn how to count the # of people entering & leavi… https://t.co/JQYXfNxrnu
- Adrian Rosebrock (@PyImageSearch) August 13, 2018
Nice to see Netflix doing stuff like this
RT @MichelleUfford: At Netflix we're reimagining what a Jupyter notebook can be, who can use it, & what you can do with it. Check out my Ne…
- Enthought Inc. (@enthought) August 18, 2018
Deep Learning basics by Sentdex!
Just posted an updated Deep Learning basics with Python, TensorFlow and Keras tutorial: https://t.co/Vr3nvT7CRo… https://t.co/AqLE0jMaRG
- Harrison Kinsley🐍 (@Sentdex) August 11, 2018
This is a great security step. Nice going PyPI!
RT @pypi: PyPI will no longer accept passwords that have been published in data breaches. For background you can take a look at https://t.…
- Ewa Jodlowska (@ewa_jodlowska) August 14, 2018
This is why the UX is so important.
"The average user doesn't give a damn what happens, as long as (1) it works and (2) it's fast." - Daniel J. Bernstein
- Programming Wisdom (@CodeWisdom) August 17, 2018
Altair version 2.2 released
Altair version 2.2 just released! This adds support to Vega-Lite 2.6, as well as offering a number of API improveme… https://t.co/5oSmcCA5Pq
- Jake VanderPlas (@jakevdp) August 14, 2018
ripgrep is faster than {grep, ag, git grep, ucg, pt, sift} https://t.co/gTUnxyTMw2
- Regular Expression (@RegexTip) August 16, 2018
Django v Wordpress
Django vs WordPress: Learn how to decide between these 2 very different #ContentManagement systems:… https://t.co/DP1QdmFxrP
- Caktus Group (@CaktusGroup) August 17, 2018
How to use Bootstrap 4 forms in Django
RT @pybites: How to Use Bootstrap 4 Forms With #Django https://t.co/iL9gMxy8D3 via @vitorfs
- Martin Uribe (@clamytoe) August 15, 2018
PyConAU is all sold out! Who's going next week?
That's it! We're sold out! There are still some spaces left at our development sprints for main conference attende… https://t.co/WqyipT9dq6
- PyCon Australia (@pyconau) August 17, 2018
Python text-to-speech!
Text to speech conversion using python using gtts https://t.co/zyeDeIyiwJ
- Python LibHunt (@PythonLibHunt) August 18, 2018
Nice! A web UI for pdb!
Looks useful.. python-web-pdb: Web-based remote UI for Python's PDB debugger https://t.co/J4WgKJ3EuI
- Ned Batchelder (@nedbat) August 07, 2018
>>> from pybites import Bob, Julian Keep Calm and Code in Python!
19 Aug 2018 10:44am GMT
Evennia: Inline building in upcoming Evennia 0.8
Evennia, the Python MUD-server game development kit, is slowly creeping closer to its 0.8 release.
In our development branch I've just pushed the first version of the new OLC (OnLine Creator) system. This is a system to allow builders (who may have limited coding knowledge) to customize and spawn new in-game objects more easily without code access. It's started with the olc command in-game. This is a visual system for manipulating Evennia Prototypes.
Briefly on Prototypes
The Prototype is an Evennia concept that has been around a good while. The prototype is a Python dictionary that holds specific keys with values representing properties on a game object. Here's an example of a simple prototype:
{"key": "My house",
"typeclass": "typeclasses.houses.MyHouse"}
By passing this dict to the spawner, a new object named "My house" will be created. It will be set up with the given typeclass (a 'typeclass' is, in Evennia lingo, a Python class with a database backend). A prototype can specify all aspects of an in-game object - its attributes (like description and other game-specific properties), tags, aliases, location and so on. Prototypes also support inheritance - so you can expand on an existing template without having to add everything fresh every time.
There are two main reasons for the Prototypes existing in Evennia:
- They allow you to customize individual objects easier. For example you could have a 'Goblin' base prototype with inheriting prototypes Goblin Wizard" and "Goblin Chieftain" - all using the same Typeclass, but with different Attributes, equipment etc.
- Prototypes can be manipulated and scripted by builders without needing full Python access. This means that while the Typeclasses are designed and supplied by the Python developer, the builders can then use that typeclass to make very different types of object instances in-game.
What's new
As said, Prototypes have been around for a good while in Evennia. But in the past they were either manually entered directly as a dict on the command line, or created in code and read from a Python module. The former solution is cumbersome and requires that you know how to build a proper-syntax Python dictionary. The latter requires server code access, making them less useful to builders than they could be.
Note: If you are visually impaired, each image is also a link to a text-only version.
In Evennia 0.8, while you can still insert the Prototype as a raw dict, spawn/menu or the new olc command opens a new menu-driven interface.
More importantly, builders can now create, save and load prototypes in the database for themselves and other builders to use. The prototypes can be tagged and searched as a joint resource. Builders can also lock prototypes if others are not to be able to read or use them to spawn things. Developers can still supply module-based "read-only" prototypes (for use as starting points or examples to their Builders, for example).
You can now also use the menu to search for and create a new Prototype based on an existing object (if you have access to do so). This makes it quick to start up a new prototype and tweak it for spawning other similar objects. Of course you could spawn temporary objects without saving the prototype as well.
![The Typeclass defines what 'type' of object this is - the actual working code to use. All spawned objects must have a typeclass. If not given here, the typeclass must be set in one of the prototype's parents. [No typeclass set] ______________________________________________________________________________________________________________________________________________ Back (prototype-parent) | Forward (key) | Index | Validate prototype | Quit 1: evennia.contrib.tutorial_world.mob.Mob 7: evennia.contrib.tutorial_world.objects.TutorialObject 2: evennia.contrib.tutorial_world.objects.Climbable 8: evennia.contrib.tutorial_world.objects.Weapon 3: evennia.contrib.tutorial_world.objects.CrumblingWall 9: evennia.contrib.tutorial_world.objects.WeaponRack 4: evennia.contrib.tutorial_world.objects.LightSource 10: evennia.contrib.tutorial_world.rooms.BridgeRoom 5: evennia.contrib.tutorial_world.objects.Obelisk current: (1/3) 6: evennia.contrib.tutorial_world.objects.Readable next page](https://1.bp.blogspot.com/-DYcYStEWXKk/W3f_-bnH4vI/AAAAAAAAJJY/iv4-GN8NTpUCpBvyXBwMUABG8TTPJYN9ACLcBGAs/s400/Screenshot%2Bfrom%2B2018-08-18%2B13-15-44.png "Typeclass selection, miltipage")
Builders will likely not know which typeclasses are available in the code base. There are new a few ways to list them. The menu display makes use of Evennia 0.8's new EvMenu improvements, which allows for automatically creating multi-page listings (see example above).
There is also a new switch to the typeclass command, /list, that will list all available typeclasses outside of the OLC.
Protfuncs
Another new feature are Protfuncs. Similarly to how Inlinefuncs allows for calling for the result of a function call inside a text string, Protfuncs allows for calling functions inside a prototype's values. It's given on the form $funcname(arguments), where arguments could themselves contain one or more nested Protfuncs.
As with other such systems in Evennia, only Python functions in a specific module or modules (given by settings) are available for use as Protfuncs in-game. A bunch of default ones are included out of the box. Protfuncs are called at the time of spawning. So for example, you could set the Attribute
Strength = $randint(5, 20)
to automatically spawn objects with a random strength between 5 and 20.
When spawning, the olc will validate the prototype and run tests on any Protfunc used. For convenience you can override the spawn-location if any is hard-coded in the prototype.
The system will also allow you to try updating existing objects created from the same-named prototype earlier. It will sample the existing objects and calculate a 'diff' to apply. This is bit is still a bit iffy, with edge cases that still needs fixing.
Current status
The OLC is currently in the develop branch of Evennia - what will soon(ish) merge to become Evennia 0.8.
It's a pretty big piece of code and as such it's still a bit unstable and there are edge cases and display issues to fix. But it would be great with more people trying it out and reporting errors so the childhood issues can be ironed out before release!
Building Image: Released as Creative Commons here
19 Aug 2018 10:40am GMT
Evennia: Inline building in upcoming Evennia 0.8
Evennia, the Python MUD-server game development kit, is slowly creeping closer to its 0.8 release.
In our development branch I've just pushed the first version of the new OLC (OnLine Creator) system. This is a system to allow builders (who may have limited coding knowledge) to customize and spawn new in-game objects more easily without code access. It's started with the olc command in-game. This is a visual system for manipulating Evennia Prototypes.
Briefly on Prototypes
The Prototype is an Evennia concept that has been around a good while. The prototype is a Python dictionary that holds specific keys with values representing properties on a game object. Here's an example of a simple prototype:
{"key": "My house",
"typeclass": "typeclasses.houses.MyHouse"}
By passing this dict to the spawner, a new object named "My house" will be created. It will be set up with the given typeclass (a 'typeclass' is, in Evennia lingo, a Python class with a database backend). A prototype can specify all aspects of an in-game object - its attributes (like description and other game-specific properties), tags, aliases, location and so on. Prototypes also support inheritance - so you can expand on an existing template without having to add everything fresh every time.
There are two main reasons for the Prototypes existing in Evennia:
- They allow you to customize individual objects easier. For example you could have a 'Goblin' base prototype with inheriting prototypes Goblin Wizard" and "Goblin Chieftain" - all using the same Typeclass, but with different Attributes, equipment etc.
- Prototypes can be manipulated and scripted by builders without needing full Python access. This means that while the Typeclasses are designed and supplied by the Python developer, the builders can then use that typeclass to make very different types of object instances in-game.
What's new
As said, Prototypes have been around for a good while in Evennia. But in the past they were either manually entered directly as a dict on the command line, or created in code and read from a Python module. The former solution is cumbersome and requires that you know how to build a proper-syntax Python dictionary. The latter requires server code access, making them less useful to builders than they could be.
Note: If you are visually impaired, each image is also a link to a text-only version.
In Evennia 0.8, while you can still insert the Prototype as a raw dict, spawn/menu or the new olc command opens a new menu-driven interface.
More importantly, builders can now create, save and load prototypes in the database for themselves and other builders to use. The prototypes can be tagged and searched as a joint resource. Builders can also lock prototypes if others are not to be able to read or use them to spawn things. Developers can still supply module-based "read-only" prototypes (for use as starting points or examples to their Builders, for example).
You can now also use the menu to search for and create a new Prototype based on an existing object (if you have access to do so). This makes it quick to start up a new prototype and tweak it for spawning other similar objects. Of course you could spawn temporary objects without saving the prototype as well.
Builders will likely not know which typeclasses are available in the code base. There are new a few ways to list them. The menu display makes use of Evennia 0.8's new EvMenu improvements, which allows for automatically creating multi-page listings (see example above).
There is also a new switch to the typeclass command, /list, that will list all available typeclasses outside of the OLC.
Protfuncs
Another new feature are Protfuncs. Similarly to how Inlinefuncs allows for calling for the result of a function call inside a text string, Protfuncs allows for calling functions inside a prototype's values. It's given on the form $funcname(arguments), where arguments could themselves contain one or more nested Protfuncs.
As with other such systems in Evennia, only Python functions in a specific module or modules (given by settings) are available for use as Protfuncs in-game. A bunch of default ones are included out of the box. Protfuncs are called at the time of spawning. So for example, you could set the Attribute
Strength = $randint(5, 20)
to automatically spawn objects with a random strength between 5 and 20.
When spawning, the olc will validate the prototype and run tests on any Protfunc used. For convenience you can override the spawn-location if any is hard-coded in the prototype.
The system will also allow you to try updating existing objects created from the same-named prototype earlier. It will sample the existing objects and calculate a 'diff' to apply. This is bit is still a bit iffy, with edge cases that still needs fixing.
Current status
The OLC is currently in the develop branch of Evennia - what will soon(ish) merge to become Evennia 0.8.
It's a pretty big piece of code and as such it's still a bit unstable and there are edge cases and display issues to fix. But it would be great with more people trying it out and reporting errors so the childhood issues can be ironed out before release!
Building Image: Released as Creative Commons here
19 Aug 2018 10:40am GMT
Evennia: Evennia in pictures
This article describes the MU* development system Evennia using pictures!
Figure 1: The parts of the Evennia library
Figure 2: The full setup for mygame
Figure 3: The Database Abstraction of Evennia entities
In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.
Figure 5: Adding persistent Attributes to a game entity.
The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The "strength" and "skills" Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.
Figure 6: Sessions, Players and Objects
Figure 7: Commands are Python classes too
Figure 8: Commands are grouped together in sets and always associated with game entities.
Figure 9: Command Sets can affect those around them
Trigger's Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We'll explain why below.
Figure 10: The name Command "Set" is not just a name
Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.
19 Aug 2018 9:30am GMT
Evennia: Evennia in pictures
This article describes the MU* development system Evennia using pictures!
Figure 1: The parts of the Evennia library
Figure 2: The full setup for mygame
Figure 3: The Database Abstraction of Evennia entities
In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.
Figure 5: Adding persistent Attributes to a game entity.
The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The "strength" and "skills" Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.
Figure 6: Sessions, Players and Objects
Figure 7: Commands are Python classes too
Figure 8: Commands are grouped together in sets and always associated with game entities.
Figure 9: Command Sets can affect those around them
Trigger's Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We'll explain why below.
Figure 10: The name Command "Set" is not just a name
Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.
19 Aug 2018 9:30am GMT
Kushal Das: Aadhaar, the mass surveillance system
If you are following me on Twitter, you have already seen a lot of (re)tweets related to Aadhaar. For the people first time hearing this term, it is a 12 digit unique identification number provided by the Unique Identification Authority of India (UIDAI). It is also the world's largest bio-metric ID system. It is supposed to be a voluntary service.
From the very beginning, this project tried to hide the details from the Indian citizens. Let it be privacy advocates or security researchers or human rights activists, everyone predicted that this will become a monster, a mass surveillance system, a tool of choice of the power hungry dictators.
Like any other complex system, the majority of the people only see the advertisements from the government and completely miss all the problems and horror stories this project is creating. Here are a few links below for the interested people to read.
- IRC session log from Kiran Jonnalagadda
- Karana (detailed posts related to various parts of the project) Must read site.
- Aadhaar fail Details of scams/failures/deaths/other issues from Aadhaar project
Neither my wife, nor our daughter has an Aadhaar (I also don't have one), that means Py (our daughter) did not get admission to any school last year.
Whenever security researchers or journalists tried to report on the project, the UIDAI tried to hide behind denials and police complaints against the journalists or researchers. There are various reports on how one can get access (both read/write) to the actual production database with as little as $10-30. We now have examples of terrorist organizations having access to the same database. The UIDAI kept telling how this is an unhackable technology and for security they have a 13 feet wall outside of the data center which in turn will keep all hackers away.
They have already build 360 degree databases on top of Aadhaar, and now they are trying to link DNA to the same system.
The current government of India tried their level best to argue in the Supreme Court of India to tell that Indians don't have any rights to privacy. But, thankfully they failed in this effort, and the Supreme Court ruled privacy as a fundamental right. We are now waiting for the judgment on the Aadhaar (which will hopefully come out in the next few weeks).
Meanwhile, the evil nexus is pushing down Aadhaar to the throats of the Indian citizens and Pakistani spies and gods.
A few days ago, in an event in Jaipur, they asked Edward Snowden the following question.
How big of an issue is privacy?
The answer started with from where that argument comes from.
The answer is that Nazi Germany. The nazi minister of propaganda Joseph Goebbels did this. Because he was trying to change the conversation away from "What are your rights?" and "What evidences must the government show?" to violet them, to intrude into your private life and instead said "Why do you need your rights?", "How can you justify your rights?", "Isn't strange that you are invoking your rights? Isn't that unusual?". But, in a free society this is the opposite of the way it is supposed to work. We don't need to explain why you have a right. You don't need to explain why it is valuable, why you need it. It is for the government to explain why you don't deserve it. They go to a court, they show that you are a criminal. This is increasingly falling out of favor, because the governments and companies think that it is inefficient. It is too much work. Life would be easier, life would be more convenient for them, life would be more profitable for them if we didn't have any rights at all.
But, privacy isn't about something to hide, privacy is about something to protect. And that is the very concept of liberty. It is the idea that there can be some part of you, of your life, of your ideas that belong to you, not to society. And you get to make the decision about who you share that with. -- Edward Snowden
Why are we reading this in your blog?
This might a question for many of you. Why are reading this in a blog post or in a planet? Because we, the people with the knowledge of technology are also part of these evil plans. We now know about many private companies taking part with their local government to build 360 degree profiles, to track the citizens and to run the mass surveillance systems. For example, related to Aadhaar, for the last 4 years, Google silently pushed the Aadhaar support phone number (which now UIDAI is trying to stay away from) to every Google Android phone in India. When they got caught red handed, they claimed that they did it inadvertently. Finacle software by Infosys denies creation of bank accounts without Aadhaar. Microsoft is working to link Skype with Aadhaar. Bill Gates is trying to push the idea that Aadhaar is all good, and does not have any issues.
What can you do?
You can start by educating yourself first. Read more about the technologies which controls our lives. Have doubt about the things and try to understand how they actually work. Write about them, ask questions to the people in power. Talk about the issues to your friends and family.
This is not gong to be an easy task, but, we all should keep fighting back to make sure of a better future for our next generation.
19 Aug 2018 6:33am GMT
Kushal Das: Aadhaar, the mass surveillance system
If you are following me on Twitter, you have already seen a lot of (re)tweets related to Aadhaar. For the people first time hearing this term, it is a 12 digit unique identification number provided by the Unique Identification Authority of India (UIDAI). It is also the world's largest bio-metric ID system. It is supposed to be a voluntary service.
From the very beginning, this project tried to hide the details from the Indian citizens. Let it be privacy advocates or security researchers or human rights activists, everyone predicted that this will become a monster, a mass surveillance system, a tool of choice of the power hungry dictators.
Like any other complex system, the majority of the people only see the advertisements from the government and completely miss all the problems and horror stories this project is creating. Here are a few links below for the interested people to read.
- IRC session log from Kiran Jonnalagadda
- Karana (detailed posts related to various parts of the project) Must read site.
- Aadhaar fail Details of scams/failures/deaths/other issues from Aadhaar project
Neither my wife, nor our daughter has an Aadhaar (I also don't have one), that means Py (our daughter) did not get admission to any school last year.
Whenever security researchers or journalists tried to report on the project, the UIDAI tried to hide behind denials and police complaints against the journalists or researchers. There are various reports on how one can get access (both read/write) to the actual production database with as little as $10-30. We now have examples of terrorist organizations having access to the same database. The UIDAI kept telling how this is an unhackable technology and for security they have a 13 feet wall outside of the data center which in turn will keep all hackers away.
They have already build 360 degree databases on top of Aadhaar, and now they are trying to link DNA to the same system.
The current government of India tried their level best to argue in the Supreme Court of India to tell that Indians don't have any rights to privacy. But, thankfully they failed in this effort, and the Supreme Court ruled privacy as a fundamental right. We are now waiting for the judgment on the Aadhaar (which will hopefully come out in the next few weeks).
Meanwhile, the evil nexus is pushing down Aadhaar to the throats of the Indian citizens and Pakistani spies and gods.
A few days ago, in an event in Jaipur, they asked Edward Snowden the following question.
How big of an issue is privacy?
The answer started with from where that argument comes from.
The answer is that Nazi Germany. The nazi minister of propaganda Joseph Goebbels did this. Because he was trying to change the conversation away from "What are your rights?" and "What evidences must the government show?" to violet them, to intrude into your private life and instead said "Why do you need your rights?", "How can you justify your rights?", "Isn't strange that you are invoking your rights? Isn't that unusual?". But, in a free society this is the opposite of the way it is supposed to work. We don't need to explain why you have a right. You don't need to explain why it is valuable, why you need it. It is for the government to explain why you don't deserve it. They go to a court, they show that you are a criminal. This is increasingly falling out of favor, because the governments and companies think that it is inefficient. It is too much work. Life would be easier, life would be more convenient for them, life would be more profitable for them if we didn't have any rights at all.
But, privacy isn't about something to hide, privacy is about something to protect. And that is the very concept of liberty. It is the idea that there can be some part of you, of your life, of your ideas that belong to you, not to society. And you get to make the decision about who you share that with. -- Edward Snowden
Why are we reading this in your blog?
This might a question for many of you. Why are reading this in a blog post or in a planet? Because we, the people with the knowledge of technology are also part of these evil plans. We now know about many private companies taking part with their local government to build 360 degree profiles, to track the citizens and to run the mass surveillance systems. For example, related to Aadhaar, for the last 4 years, Google silently pushed the Aadhaar support phone number (which now UIDAI is trying to stay away from) to every Google Android phone in India. When they got caught red handed, they claimed that they did it inadvertently. Finacle software by Infosys denies creation of bank accounts without Aadhaar. Microsoft is working to link Skype with Aadhaar. Bill Gates is trying to push the idea that Aadhaar is all good, and does not have any issues.
What can you do?
You can start by educating yourself first. Read more about the technologies which controls our lives. Have doubt about the things and try to understand how they actually work. Write about them, ask questions to the people in power. Talk about the issues to your friends and family.
This is not gong to be an easy task, but, we all should keep fighting back to make sure of a better future for our next generation.
19 Aug 2018 6:33am GMT
18 Aug 2018
Planet Python
Karim Elghamrawy: Python Lambdas Explained (With Examples)
In this article, I will teach you exactly what a python lambda is. As a matter of fact if you know what functions are and how to define functions in Python then you already know what a lambda is. A Python lambda is just a Python function. But may be like a special type of [...]
The post Python Lambdas Explained (With Examples) appeared first on Afternerd.
18 Aug 2018 11:25pm GMT
Karim Elghamrawy: Python Lambdas Explained (With Examples)
In this article, I will teach you exactly what a python lambda is. As a matter of fact if you know what functions are and how to define functions in Python then you already know what a lambda is. A Python lambda is just a Python function. But may be like a special type of [...]
The post Python Lambdas Explained (With Examples) appeared first on Afternerd.
18 Aug 2018 11:25pm GMT
Codementor: How and why I built IVR enabled AWS cloud environment
Cloud infra management and automation using IVR system.
18 Aug 2018 6:02pm GMT
Codementor: How and why I built IVR enabled AWS cloud environment
Cloud infra management and automation using IVR system.
18 Aug 2018 6:02pm GMT
Marc Richter: Create your own Telegram bot with Django on Heroku – Part 4 – pull vs. push method
In the previous part of this series, we started to get familiar with telepot, a Python module to interact with Telegram bots and had a short look at how the Telegram bot API is providing messages as JSON structures.
Today we will talk about the Webhook-method (push) instead of the previously introduced getUpdate-method (pull).
What's the difference in getUpdates and a webhook? 
The Default: getUpdates (pull) 
By default (when a new bot is registered), a bot is configured to cache messages until they are actively fetched by some request (getUpdates). This is also called "pull method".
When a message is sent to the bot while it is configured for this mode, it is cached in some kind of buffer within the infrastructure of Telegram (until they are delivered to the bot, but not longer than 24 hours, as described in the docs).
While this mode offers an easy start because you do not need to have any infrastructure or code prepared to create the bot and start exchanging messages with it, it has some downsides:
- You have to take care of having your code to ask for updates frequently.
- You will often ask for updates while there is none, generating unnecessary load and traffic.
- You do not have the opportunity to implement a real-time service since updates will always be delayed until the next update execution.
There might be a good reason to keep it like this nevertheless, like if you do not have a way to meet the technical requirements for the alternative method which is a webhook, like that you do not have a way to expose an interface to the public internet for example or if you want to prepare and test some code before firing up a deployed service. But most serious bots probably will change this to the alternate webhook (push) method sooner or later.
GET and POST HTTP methods
Before we look at the webhook push method, let's first refresh some basics about how HTTP works. This will only cover some basics to better understand what is going on in the next sections. There is more to say and explain about the methods shown here and there are more methods to the protocol as well; not because of no reason there are 656 pages to O'Reilly's book "HTTP: The Definitive Guide". Since obviously, this is beyond the scope of this article, if you want to know more, please help yourself.
GET
When you are surfing the web with your favorite browser, GET requests are sent by that browser to the web servers providing your favorite websites, asking for a specific resource (like "
GET / HTTP/1.1
" is asking for the page index). To fetch the article you are currently reading, your browser most certainly sent something like this to my web server:
"GET /create-your-own-telegram-bot-with-django-on-heroku-part-3/?somethingIWantToSet=HubbaHubba HTTP/1.1"
followed by a myriad of additional requests for each asset (like CSS files, JS, images, …). The web server then does it's magic and delivers the associated content in its answer; classically: A webpage.
The "?" in that request string separates the URL from an optional, additional section in that GET request: The query-string (
?somethingIWantToSet=HubbaHubba
). Even though GET is a method to request data from an HTTP resource, data can be sent to it, this way. Since this becomes quite unhandy soon and everyone can easily read and manipulate the content of the data sent to the server, this is only recommendable for really short things, like selecting a bright or dark theme for the page you are requesting like this:
http://www.my-great-site.com/index.php?theme=dark
To sum this up: GET is used to request data from a specified resource.
POST
Nowadays, most websites are not delivering just static content anymore. Instead, you can interact with them. You can, for example, send an email by filling a contact form or publish an answer to some forum's thread or similar. To make this work, you need to send data to the remote website instead of sending a request to the remote web server to have something sent back to you. The method in the HTTP protocol enabling you to do so is called POST.
If, for example, somebody is sending me an email using the contact form on my website, I can find a request like this in the access logs of my server:
162.158.89.81 - - [18/Aug/2018:13:11:30 +0200] "POST /contact/ HTTP/1.1" 200 21185 "https://www.marc-richter.info/contact/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0"
There is some detail in that, like the HTTP status (
200
) indicating that the request was successful, the referer of the request (
https://www.marc-richter.info/contact/
) telling where the user came from, some details about the user agent (
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
) telling what OS and Webbrowser the visitor was using, etc. But the important part for us is:
POST /contact/ HTTP/1.1
This looks pretty much the same as the GET request is, isn't it?
… wait a moment: Has somebody sent you an empty mail?
No. In fact, I sent it to myself with the following content:
To: Marc Richter
Name: Marc Richter
Email: noreply@marc-richter.info
Subject: TestmailMessage:
Hi Marc,
great site!
BR
The reason why you do not see this in the logline before is not that I cut that part. It doesn't show up there, because unlike with GET requests, the POST request is initiating the communication with calling for the destination path only, without any additional in-URL data visible. The content/payload of the request is not part of the initializing requests URL, but is appended to the header of the request.
This is not only more secure to exchange data with the server, but also it is more convenient to exchange bigger amounts of data this way. Imagine, that all data which is ever sent to your favorite social network can be read by any admin who can read the log files of that server … or your employer since you are routed over his proxy servers … 
To understand webhooks and to sum this up: Telegram bots are sending the JSON data we already saw in the previous part of this series to any URL we configure to them, using POST requests. The JSON data is then part of these request's payload and can easily be extracted by Django (or any other application).
Again: If you want to get additional details on this, look them up for yourself; to understand how webhooks are working, what I explained so far is already sufficient to follow this article.
Webhook (push) 
Now that we know what POST is and how it is working, let's talk about how webhooks work.
There is a section in the official bot guide of Telegram, which explains how Webhooks work in three different levels of details; if you want to understand more details about Webhooks, I recommend reading at least the shortest version of it; I think it really is a great start for anybody not familiar with the concept of a webhook.
In my own words: A webhook is primarily a normal HTTP(S) socket, provided by a web server - like software listening to it for HTTP requests. Some piece of software is picking up everything which is sent towards this interface and applies some pre-defined logic on the data received. What logic applies is defined by your code, picking up that data.
This happens in real-time and is aware of the delivery result: If the message could not be forwarded to the defined interface, maybe because the server is down or there are temporary connection issues, Telegram caches the messages just like it does with the getUpdate pull method and retries to deliver it again after some time. As soon as the message was handed over to your interface successfully, Telegram forgets about the message and will never re-transmit it to your bot again.
Giving webhooks a try
Enough of the boring theory: I bet you are desperate to see this whole stuff in action, don't you? Let's change your bot to webhook-mode and have everything sent to it forwarded to examine what we need to prepare our bot application for. After that, we will switch back to the getUpdates pull method again.
Attention: Please be aware that everything that will be sent to the bot will be forwarded to a 3rd party's web service, I do have no control of. Please make sure you are OK with any data privacy policy of your country and that service before following the next lines, since I do not take responsibility about these things (if you revealed your online banking TAN lists because you felt like your bot could be interested in them or your partner is sending nudes unaware of this change, for example 
).
First, you need to visit https://webhook.site in your favorite browser. You will be presented a page telling you all the details to utilize a unique webhook which just got created for you automatically without any registration - pretty awesome, isn't it?
Copy that link and click the "Open in a new tab"-link once. That will open a new tab, showing an empty page; but - wait! The welcome page of your webhook will have changed immediately, listing a new GET request to have just arrived. That was your browser opening that empty page.
It works the same with POST or any other HTTP method anybody makes towards that interface.
Let's now configure our bot's webhook to use that URL you just copied as a destination interface for it's webhook. You will need two things:
- WEBHOOK_URL: The URL of the webhook you just created
- BOT_TOKEN: The Token of your bot, we created in the previous sections of this series and I advised you to take a note on it because we will need that later; now is "later"
In your browser, you now navigate to the following URL after you have crafted the full URL:
https://api.telegram.org/bot{BOT_TOKEN}/setWebhook?url={WEBHOOK_URL}
You need to replace the two "
{}
"-markers with your own data. You should be presented something like this:
{"ok":true,"result":true,"description":"Webhook was set"}
And: You are good to go!
Make sure the page of your webhook is opened in your browser and then, send something to your bot in Telegram. Immediately (!) you should see that new message as a POST request in the "requests" bar on the left of your webhook page. You can tick the "Format JSON" checkbox at the upper right corner of that page to make the JSON data a bit easier to read.
And that's it! This is a preview of how the messages which are sent to your bot will be forwarded to your webhook.
Telegram Webhook representation
Go on, play around with that. Have different datatypes (like images, text, vcards) sent to your bot and check out how it will arrive at the webhook.
You can rely on this format will be the same which are about to hit your Python code, soon. Better be prepared for everything 
Switching back to getUpdate pull method
When you are done, better switch back to the getUpdate pull method, to not accidentally expose sensitive data to this publically available service.
This can be done, sending another GET request to the bot's API like we used to enable the hook. The only difference is that we won't provide a webhook URL this time:
https://api.telegram.org/bot{BOT_TOKEN}/setWebhook?url=
Success should be indicated by the following message:
{"ok":true,"result":true,"description":"Webhook was deleted"}
Outlook for the next part of the series
So far so good! For this part of the series, that's it again!
We just learned how webhooks are working and how to enable your bot to make use of that method, how to analyze the data received by it on any webhook and how to disable it again, re-enabling getUpdate pull method.
Next time, I will show you how to prepare Heroku to host the Django app we will create in later chapters and already upload a first demo site to it.
I hope you enjoyed this part! Please let me know of all the things you either enjoyed or did not like that much and how I can do it better in the comments.
Born in 1982, Marc Richter is an IT enthusiastic since 1994. He became addicted when he first put hands on their family's pc and never stopped investigating and exploring new things since then.
He is married to Jennifer Richter and proud father of two wonderful children, Lotta and Linus.
His current professional focus is DevOps and Python development.
An exhaustive bio can be found at this blog post.
Found my articles useful? Maybe you would like to support my efforts and give me a tip then?
Der Beitrag Create your own Telegram bot with Django on Heroku - Part 4 - pull vs. push method erschien zuerst auf Marc Richter's personal site.
18 Aug 2018 12:46pm GMT
Marc Richter: Create your own Telegram bot with Django on Heroku – Part 4 – pull vs. push method
In the previous part of this series, we started to get familiar with telepot, a Python module to interact with Telegram bots and had a short look at how the Telegram bot API is providing messages as JSON structures.
Today we will talk about the Webhook-method (push) instead of the previously introduced getUpdate-method (pull).
What's the difference in getUpdates and a webhook?
The Default: getUpdates (pull)
By default (when a new bot is registered), a bot is configured to cache messages until they are actively fetched by some request (getUpdates). This is also called "pull method".
When a message is sent to the bot while it is configured for this mode, it is cached in some kind of buffer within the infrastructure of Telegram (until they are delivered to the bot, but not longer than 24 hours, as described in the docs).
While this mode offers an easy start because you do not need to have any infrastructure or code prepared to create the bot and start exchanging messages with it, it has some downsides:
- You have to take care of having your code to ask for updates frequently.
- You will often ask for updates while there is none, generating unnecessary load and traffic.
- You do not have the opportunity to implement a real-time service since updates will always be delayed until the next update execution.
There might be a good reason to keep it like this nevertheless, like if you do not have a way to meet the technical requirements for the alternative method which is a webhook, like that you do not have a way to expose an interface to the public internet for example or if you want to prepare and test some code before firing up a deployed service. But most serious bots probably will change this to the alternate webhook (push) method sooner or later.
GET and POST HTTP methods
Before we look at the webhook push method, let's first refresh some basics about how HTTP works. This will only cover some basics to better understand what is going on in the next sections. There is more to say and explain about the methods shown here and there are more methods to the protocol as well; not because of no reason there are 656 pages to O'Reilly's book "HTTP: The Definitive Guide". Since obviously, this is beyond the scope of this article, if you want to know more, please help yourself.
GET
When you are surfing the web with your favorite browser, GET requests are sent by that browser to the web servers providing your favorite websites, asking for a specific resource (like "
GET / HTTP/1.1
" is asking for the page index). To fetch the article you are currently reading, your browser most certainly sent something like this to my web server:
"GET /create-your-own-telegram-bot-with-django-on-heroku-part-3/?somethingIWantToSet=HubbaHubba HTTP/1.1"
followed by a myriad of additional requests for each asset (like CSS files, JS, images, …). The web server then does it's magic and delivers the associated content in its answer; classically: A webpage.
The "?" in that request string separates the URL from an optional, additional section in that GET request: The query-string (
?somethingIWantToSet=HubbaHubba
). Even though GET is a method to request data from an HTTP resource, data can be sent to it, this way. Since this becomes quite unhandy soon and everyone can easily read and manipulate the content of the data sent to the server, this is only recommendable for really short things, like selecting a bright or dark theme for the page you are requesting like this:
http://www.my-great-site.com/index.php?theme=dark
To sum this up: GET is used to request data from a specified resource.
POST
Nowadays, most websites are not delivering just static content anymore. Instead, you can interact with them. You can, for example, send an email by filling a contact form or publish an answer to some forum's thread or similar. To make this work, you need to send data to the remote website instead of sending a request to the remote web server to have something sent back to you. The method in the HTTP protocol enabling you to do so is called POST.
If, for example, somebody is sending me an email using the contact form on my website, I can find a request like this in the access logs of my server:
162.158.89.81 - - [18/Aug/2018:13:11:30 +0200] "POST /contact/ HTTP/1.1" 200 21185 "https://www.marc-richter.info/contact/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0"
There is some detail in that, like the HTTP status (
200
) indicating that the request was successful, the referer of the request (
https://www.marc-richter.info/contact/
) telling where the user came from, some details about the user agent (
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
) telling what OS and Webbrowser the visitor was using, etc. But the important part for us is:
POST /contact/ HTTP/1.1
This looks pretty much the same as the GET request is, isn't it?
… wait a moment: Has somebody sent you an empty mail?
No. In fact, I sent it to myself with the following content:
To: Marc Richter
Name: Marc Richter
Email: noreply@marc-richter.info
Subject: TestmailMessage:
Hi Marc,
great site!
BR
The reason why you do not see this in the logline before is not that I cut that part. It doesn't show up there, because unlike with GET requests, the POST request is initiating the communication with calling for the destination path only, without any additional in-URL data visible. The content/payload of the request is not part of the initializing requests URL, but is appended to the header of the request.
This is not only more secure to exchange data with the server, but also it is more convenient to exchange bigger amounts of data this way. Imagine, that all data which is ever sent to your favorite social network can be read by any admin who can read the log files of that server … or your employer since you are routed over his proxy servers …
To understand webhooks and to sum this up: Telegram bots are sending the JSON data we already saw in the previous part of this series to any URL we configure to them, using POST requests. The JSON data is then part of these request's payload and can easily be extracted by Django (or any other application).
Again: If you want to get additional details on this, look them up for yourself; to understand how webhooks are working, what I explained so far is already sufficient to follow this article.
Webhook (push)
Now that we know what POST is and how it is working, let's talk about how webhooks work.
There is a section in the official bot guide of Telegram, which explains how Webhooks work in three different levels of details; if you want to understand more details about Webhooks, I recommend reading at least the shortest version of it; I think it really is a great start for anybody not familiar with the concept of a webhook.
In my own words: A webhook is primarily a normal HTTP(S) socket, provided by a web server - like software listening to it for HTTP requests. Some piece of software is picking up everything which is sent towards this interface and applies some pre-defined logic on the data received. What logic applies is defined by your code, picking up that data.
This happens in real-time and is aware of the delivery result: If the message could not be forwarded to the defined interface, maybe because the server is down or there are temporary connection issues, Telegram caches the messages just like it does with the getUpdate pull method and retries to deliver it again after some time. As soon as the message was handed over to your interface successfully, Telegram forgets about the message and will never re-transmit it to your bot again.
Giving webhooks a try
Enough of the boring theory: I bet you are desperate to see this whole stuff in action, don't you? Let's change your bot to webhook-mode and have everything sent to it forwarded to examine what we need to prepare our bot application for. After that, we will switch back to the getUpdates pull method again.
Attention: Please be aware that everything that will be sent to the bot will be forwarded to a 3rd party's web service, I do have no control of. Please make sure you are OK with any data privacy policy of your country and that service before following the next lines, since I do not take responsibility about these things (if you revealed your online banking TAN lists because you felt like your bot could be interested in them or your partner is sending nudes unaware of this change, for example ).
First, you need to visit https://webhook.site in your favorite browser. You will be presented a page telling you all the details to utilize a unique webhook which just got created for you automatically without any registration - pretty awesome, isn't it?
Copy that link and click the "Open in a new tab"-link once. That will open a new tab, showing an empty page; but - wait! The welcome page of your webhook will have changed immediately, listing a new GET request to have just arrived. That was your browser opening that empty page.
It works the same with POST or any other HTTP method anybody makes towards that interface.
Let's now configure our bot's webhook to use that URL you just copied as a destination interface for it's webhook. You will need two things:
- WEBHOOK_URL: The URL of the webhook you just created
- BOT_TOKEN: The Token of your bot, we created in the previous sections of this series and I advised you to take a note on it because we will need that later; now is "later"
In your browser, you now navigate to the following URL after you have crafted the full URL:
https://api.telegram.org/bot{BOT_TOKEN}/setWebhook?url={WEBHOOK_URL}
You need to replace the two "
{}
"-markers with your own data. You should be presented something like this:
{"ok":true,"result":true,"description":"Webhook was set"}
And: You are good to go!
Make sure the page of your webhook is opened in your browser and then, send something to your bot in Telegram. Immediately (!) you should see that new message as a POST request in the "requests" bar on the left of your webhook page. You can tick the "Format JSON" checkbox at the upper right corner of that page to make the JSON data a bit easier to read.
And that's it! This is a preview of how the messages which are sent to your bot will be forwarded to your webhook.
Telegram Webhook representation
Go on, play around with that. Have different datatypes (like images, text, vcards) sent to your bot and check out how it will arrive at the webhook.
You can rely on this format will be the same which are about to hit your Python code, soon. Better be prepared for everything
Switching back to getUpdate pull method
When you are done, better switch back to the getUpdate pull method, to not accidentally expose sensitive data to this publically available service.
This can be done, sending another GET request to the bot's API like we used to enable the hook. The only difference is that we won't provide a webhook URL this time:
https://api.telegram.org/bot{BOT_TOKEN}/setWebhook?url=
Success should be indicated by the following message:
{"ok":true,"result":true,"description":"Webhook was deleted"}
Outlook for the next part of the series
So far so good! For this part of the series, that's it again!
We just learned how webhooks are working and how to enable your bot to make use of that method, how to analyze the data received by it on any webhook and how to disable it again, re-enabling getUpdate pull method.
Next time, I will show you how to prepare Heroku to host the Django app we will create in later chapters and already upload a first demo site to it.
I hope you enjoyed this part! Please let me know of all the things you either enjoyed or did not like that much and how I can do it better in the comments.
Born in 1982, Marc Richter is an IT enthusiastic since 1994. He became addicted when he first put hands on their family's pc and never stopped investigating and exploring new things since then.
He is married to Jennifer Richter and proud father of two wonderful children, Lotta and Linus.
His current professional focus is DevOps and Python development.
An exhaustive bio can be found at this blog post.
Found my articles useful? Maybe you would like to support my efforts and give me a tip then?
Der Beitrag Create your own Telegram bot with Django on Heroku - Part 4 - pull vs. push method erschien zuerst auf Marc Richter's personal site.
18 Aug 2018 12:46pm GMT
Weekly Python StackOverflow Report: (cxxxix) stackoverflow python report
These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2018-08-18 11:06:51 GMT
- Problems with using a rough greyscale algorithm? - [37/8]
- Collatz Conjecture Python - Incorrect Output Above 2 Trillion (Only!) - [28/1]
- if-else vs "or" operation for None-check - [15/6]
- Faster Python technique to count triples from a list of numbers that are multiples of each other - [14/5]
- Python: What happens if script stops while requests.get() is executing? - [13/1]
- Extending a class by parameter in Python - [12/2]
- How do I shorten this boolean expression? - [10/6]
- Getting outer environment arguments from java using graal python - [8/1]
- Is the File-Object iterator "broken?" - [8/1]
- Swapping list elements in python where the expressions contain function calls - [8/1]
18 Aug 2018 11:07am GMT
Weekly Python StackOverflow Report: (cxxxix) stackoverflow python report
These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2018-08-18 11:06:51 GMT
- Problems with using a rough greyscale algorithm? - [37/8]
- Collatz Conjecture Python - Incorrect Output Above 2 Trillion (Only!) - [28/1]
- if-else vs "or" operation for None-check - [15/6]
- Faster Python technique to count triples from a list of numbers that are multiples of each other - [14/5]
- Python: What happens if script stops while requests.get() is executing? - [13/1]
- Extending a class by parameter in Python - [12/2]
- How do I shorten this boolean expression? - [10/6]
- Getting outer environment arguments from java using graal python - [8/1]
- Is the File-Object iterator "broken?" - [8/1]
- Swapping list elements in python where the expressions contain function calls - [8/1]
18 Aug 2018 11:07am GMT
17 Aug 2018
Planet Python
Codementor: Building a Discord Bot with Python and Repl.it
Find out how to build your own Discord bot using Python and Repl.it. We'll walk through all the steps needed to set your bot up on Discord and then code it using Python, all in the cloud.
17 Aug 2018 10:50pm GMT
Codementor: Building a Discord Bot with Python and Repl.it
Find out how to build your own Discord bot using Python and Repl.it. We'll walk through all the steps needed to set your bot up on Discord and then code it using Python, all in the cloud.
17 Aug 2018 10:50pm GMT
Davy Wybiral: DIY Game Controller (soldering project)
I got a new Hakko soldering iron so it's time to melt some metal!
This video shows how to make a working game controller out of only a perf board, some tactile buttons, and an Arduino Pro Micro. The code is available here.
17 Aug 2018 8:50pm GMT
Davy Wybiral: DIY Game Controller (soldering project)
I got a new Hakko soldering iron so it's time to melt some metal!
This video shows how to make a working game controller out of only a perf board, some tactile buttons, and an Arduino Pro Micro. The code is available here.
17 Aug 2018 8:50pm GMT
Codementor: Hunting Intermittent Tests
There's a sad time in testing land when you have a test suite that you can't rely on. Instead of running a suite and merging if it passes, you have to glance over the failed tests and say "Oh that one… that test is flaky."
17 Aug 2018 8:13pm GMT
Codementor: Hunting Intermittent Tests
There's a sad time in testing land when you have a test suite that you can't rely on. Instead of running a suite and merging if it passes, you have to glance over the failed tests and say "Oh that one… that test is flaky."
17 Aug 2018 8:13pm GMT
Bhishan Bhandari: Web Scraping with NodeJS
Web Scraping has been of an interest to a lot of businesses and individuals with the immense potential of the quantitative data available online. The data collected can entice the growth of an organization or a personal business. Through this post, we will see through examples on how NodeJS can be used to scrape content […]
The post Web Scraping with NodeJS appeared first on The Tara Nights.
17 Aug 2018 7:41pm GMT
Bhishan Bhandari: Web Scraping with NodeJS
Web Scraping has been of an interest to a lot of businesses and individuals with the immense potential of the quantitative data available online. The data collected can entice the growth of an organization or a personal business. Through this post, we will see through examples on how NodeJS can be used to scrape content […]
The post Web Scraping with NodeJS appeared first on The Tara Nights.
17 Aug 2018 7:41pm GMT
Christoph Zwerschke: Using GraphQL or REST, that is the question
This blog has been dormant for too long - it's time to post another article. In order to live up to the blog title, this article will combine some old and seasoned stuff (PostgreSQL and Shakespeare's works) with the latest trends for agile developers (GraphQL, asyncio and type hints for Python).
17 Aug 2018 4:00pm GMT
Christoph Zwerschke: Using GraphQL or REST, that is the question
This blog has been dormant for too long - it's time to post another article. In order to live up to the blog title, this article will combine some old and seasoned stuff (PostgreSQL and Shakespeare's works) with the latest trends for agile developers (GraphQL, asyncio and type hints for Python).
17 Aug 2018 4:00pm GMT
Stack Abuse: Using Regex for Text Manipulation in Python
Introduction
Text preprocessing is one of the most important tasks in Natural Language Processing (NLP). For instance, you may want to remove all punctuation marks from text documents before they can be used for text classification. Similarly, you may want to extract numbers from a text string. Writing manual scripts for such preprocessing tasks requires a lot of effort and is prone to errors. Keeping in view the importance of these preprocessing tasks, the Regular Expressions (aka Regex) have been developed in different languages in order to ease these text preprocessing tasks.
A Regular Expression is a text string that describes a search pattern which can be used to match or replace patterns inside a string with a minimal amount of code. In this tutorial, we will implement different types of regular expressions in the Python language.
To implement regular expressions, the Python's re package can be used. Import the Python's re package with the following command:
import re
Searching Patterns in a String
One of the most common NLP tasks is to search if a string contains a certain pattern or not. For instance, you may want to perform an operation on the string based on the condition that the string contains a number.
To search a pattern within a string, the match and findall function of the re package is used.
The match Function
Initialize a variable text with a text string as follows:
text = "The film Titanic was released in 1998"
Let's write a regex expression that matches a string of any length and any character:
result = re.match(r".*", text)
The first parameter of the match function is the regex expression that you want to search. Regex expression starts with the alphabet r followed by the pattern that you want to search. The pattern should be enclosed in single or double quotes like any other string.
The above regex expression will match the text string, since we are trying to match a string of any length and any character. If a match is found, the match function returns _sre.SRE_Match object as shown below:
type(result)
Output:
_sre.SRE_Match
Now to find the matched string, you can use the following command:
result.group(0)
Output:
'The film Titanic was released in 1998'
In case if no match is found by the match function, a null object is returned.
Now the previous regex expression matches a string with any length and any character. It will also match an empty string of length zero. To test this, update the value of text variable with an empty string:
text = ""
Now, if you again execute the following regex expression, a match will be found:
result = re.match(r".*", text)
Since we specified to match the string with any length and any character, even an empty string is being matched.
To match a string with a length of at least 1, the following regex expression is used:
result = re.match(r".+", text)
Here the plus sign specifies that the string should have at least one character.
Searching Alphabets
The match function can be used to find any alphabet letters within a string. Let's initialize the text variable with the following text:
text = "The film Titanic was released in 1998"
Now to find all the alphabet letter, both uppercase and lowercase, we can use the following regex expression:
result = re.match(r"[a-zA-z]+", text)
This regex expression states that match the text string for any alphabets from small a to small z or capital A to capital Z. The plus sign specifies that string should have at least one character. Let's print the match found by the above expression:
print(result.group(0))
Output:
The
In the output, you can see that the first word i.e. The is returned. This is because the match function only returns the first match found. In the regex we specified that find the patterns with both small and capital alphabets from a to z. The first match found was The. After the word The there is a space, which is not treated as an alphabet letter, therefore the matching stopped and the expression returned just The, which is the first match.
However, there is a problem with this. If a string starts with a number instead of an alphabet, the match function will return null even if there are alphabets after the number. Let's see this in action:
text = "1998 was the year when the film titanic was released"
result = re.match(r"[a-zA-z]+", text)
type(result)
Output:
NoneType
In the above script, we have updated the text variable and now it starts with a digit. We then used the match function to search for alphabets in the string. Though the text string contains alphabets, null will be returned since match function only matches the first element in the string.
To solve this problem we can use the search function.
The search Function
The search function is similar to the match function i.e. it tries to match the specified pattern. However, unlike the match function, it matches the pattern globally instead of matching only the first element. Therefore, the search function will return a match even if the string doesn't contain an alphabet at the start of the string but contains an alphabet elsewhere in the string, as shown below:
text = "1998 was the year when the film titanic was released"
result = re.search(r"[a-zA-z]+", text)
print(result.group(0))
Output:
was
The search function returns "was" since this is the first match that is found in the text string.
Matching String from the Start
To check if a string starts with a specific word, you can use the carrot key i.e. ^ followed by the word to match with the search function as shown below. Suppose we have the following string:
text = "XYZ 1998 was the year when the film titanic was released"
If we want to find out whether the string starts with "1998", we can use the search function as follows:
result = re.search(r"^1998", text)
type(result)
In the output, null will be returned since the text string doesn't contain "1998" directly at the start.
Now let's change the content text variable and add "1998" at the beginning and then check if "1998" is found at the beginning or not. Execute the following script:
text = "1998 was the year when the film titanic was released"
if re.search(r"^1998", text):
print("Match found")
else:
print("Match not found")
Output:
Match found
Matching Strings from the End
To check whether a string ends with a specific word or not, we can use the word in the regular expression, followed by the dollar sign. The dollar sign marks the end of the statement. Take a look at the following example:
text = "1998 was the year when the film titanic was released"
if re.search(r"1998$", text):
print("Match found")
else:
print("Match not found")
In the above script, we tried to find if the text string ends with "1998", which is not the case.
Output:
Match not found
Now if we update the string and add "1998" at the end of the text string, the above script will return 'Match found' as shown below:
text = "was the year when the film titanic was released 1998"
if re.search(r"1998$", text):
print("Match found")
else:
print("Match not found")
Output:
Match found
Substituting text in a String
Till now we have been using regex to find if a pattern exists in a string. Let's move forward with another advanced regex function i.e. substituting text in a string. The sub function is used for this purpose.
Let's take a simple example of the substitute function. Suppose we have the following string:
text = "The film Pulp Fiction was released in year 1994"
To replace the string "Pulp Fiction" with "Forrest Gump" (another movie released in 1994) we can use the sub function as follows:
result = re.sub(r"Pulp Fiction", "Forrest Gump", text)
The first parameter to the sub function is the regular expression that finds the pattern to substitute. The second parameter is the new text that you want as a replacement for the old text and the third parameter is the text string on which the substitute operation will be performed.
If you print the result variable, you will see the new string.
Now let's substitute all the alphabets in our string with character "X". Execute the following script:
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"[a-z]", "X", text)
print(result)
Output:
TXX XXXX PXXX FXXXXXX XXX XXXXXXXX XX XXXX 1994
It can be seen from the output that all the characters have been replaced except the capital ones. This is because we specified a-z only and not A-Z. There are two ways to solve this problem. You can either specify A-Z in the regular expression along with a-z as follows:
result = re.sub(r"[a-zA-Z]", "X", text)
Or you can pass the additional parameter flags to the sub function and set its value to re.I which refers to case insensitive, as follows:
result = re.sub(r"[a-z]", "X", text, flags=re.I)
More details about different types of flags can be found at Python regex official documentation page.
Shorthand Character Classes
There are different types of shorthand character classes that can be used to perform a variety of different string manipulation functions without having to write complex logic. In this section we will discuss some of them:
Removing Digits from a String
The regex expression to find digits in a string is \d. This pattern can be used to remove digits from a string by replacing them with an empty string of length zero as shown below:
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"\d", "", text)
print(result)
Output:
The film Pulp Fiction was released in year
Removing Alphabet Letters from a String
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"[a-z]", "", text, flags=re.I)
print(result)
Output:
1994
Removing Word Characters
If you want to remove all the word characters (letters and numbers) from a string and keep the remaining characters, you can use the \w pattern in your regex and replace it with an empty string of length zero, as shown below:
text = "The film, '@Pulp Fiction' was ? released in % $ year 1994."
result = re.sub(r"\w","", text, flags = re.I)
print(result)
Output:
, '@ ' ? % $ .
The output shows that all the numbers and alphabets have been removed.
Removing Non-Word Characters
To remove all the non-word characters, the \W pattern can be used as follows:
text = "The film, '@Pulp Fiction' was ? released in % $ year 1994."
result = re.sub(r"\W", "", text, flags=re.I)
print(result)
Output:
ThefilmPulpFictionwasreleasedinyear1994
From the output, you can see that everything has been removed (even spaces), except the numbers and alphabets.
Grouping Multiple Patterns
You can group multiple patterns to match or substitute in a string using the square bracket. In fact, we did this when we matched capital and small letters. Let's group multiple punctuation marks and remove them from a string:
text = "The film, '@Pulp Fiction' was ? released _ in % $ year 1994."
result = re.sub(r"[,@\'?\.$%_]", "", text, flags=re.I)
print(result)
Output:
The film Pulp Fiction was released in year 1994
You can see that the string in the text variable had multiple punctuation marks, we grouped all these punctuations in the regex expression using square brackets. It is important to mention that with a dot and a single quote we have to use the escape sequence i.e. backward slash. This is because by default the dot operator is used for any character and the single quote is used to denote a string.
Removing Multiple Spaces
Sometimes, multiple spaces appear between words as a result of removing words or punctuation. For instance, in the output of the last example, there are multiple spaces between in and year. These spaces can be removed using the \s pattern, which refers to a single space.
text = "The film Pulp Fiction was released in year 1994."
result = re.sub(r"\s+"," ", text, flags = re.I)
print(result)
Output:
The film Pulp Fiction was released in year 1994.
In the script above we used the expression \s+ which refers to single or multiple spaces.
Removing Spaces from Start and End
Sometimes we have a sentence that starts or ends with a space, which is often not desirable. The following script removes spaces from the beginning of a sentence:
text = " The film Pulp Fiction was released in year 1994"
result = re.sub(r"^\s+", "", text)
print(result)
Output:
The film Pulp Fiction was released in year 1994
Similarly, to remove space at the end of the string, the following script can be used:
text = "The film Pulp Fiction was released in year 1994 "
result = re.sub(r"\s+$", "", text)
print(result)
Removing a Single Character
Sometimes removing punctuation marks, such as an apostrophe, results in a single character which has no meaning. For instance, if you remove the apostrophe from the word Jacob's and replace it with space, the resultant string is Jacob s. Here the s makes no sense. Such single characters can be removed using regex as shown below:
text = "The film Pulp Fiction s was b released in year 1994"
result = re.sub(r"\s+[a-zA-Z]\s+", " ", text)
print(result)
Output:
The film Pulp Fiction was released in year 1994
The script replaces any small or capital letter between one or more spaces, with a single space.
Splitting a String
String splitting is another very important function. Strings can be split using split function from the re package. The split function returns a list of split tokens. Let's split a string of words where one or more space characters are found, as shown below:
text = "The film Pulp Fiction was released in year 1994 "
result = re.split(r"\s+", text)
print(result)
Output:
['The', 'film', 'Pulp', 'Fiction', 'was', 'released', 'in', 'year', '1994', '']
Similarly, you can use other regex expressions to split a string using the split functions. For instance, the following split function splits string of words when a comma is found:
text = "The film, Pulp Fiction, was released in year 1994"
result = re.split(r"\,", text)
print(result)
Output:
['The film', ' Pulp Fiction', ' was released in year 1994']
Finding All Instances
The match function conducts a match on the first element while the search function conducts a global search on the string and returns the first matched instance.
For instance, if we have the following string:
text = "I want to buy a mobile between 200 and 400 euros"
We want to search all the digits from this string. If we use the search function, only the first occurrence of digits i.e. 200 will be returned as shown below:
result = re.search(r"\d+", text)
print(result.group(0))
Output:
200
On the other hand, the findall function returns a list that contains all the matched utterances as shown below:
text = "I want to buy a mobile between 200 and 400 euros"
result = re.findall(r"\d+", text)
print(result)
Output:
['200', '400']
You can see from the output that both "200" and "400" is returned by the findall function.
Conclusion
In this article we studied some of the most commonly used regex functions in Python. Regular expressions are extremely useful for preprocessing text that can be further used for a variety of applications, such as topic modeling, text classification, sentimental analysis, and text summarization, etc.
17 Aug 2018 3:18pm GMT
Stack Abuse: Using Regex for Text Manipulation in Python
Introduction
Text preprocessing is one of the most important tasks in Natural Language Processing (NLP). For instance, you may want to remove all punctuation marks from text documents before they can be used for text classification. Similarly, you may want to extract numbers from a text string. Writing manual scripts for such preprocessing tasks requires a lot of effort and is prone to errors. Keeping in view the importance of these preprocessing tasks, the Regular Expressions (aka Regex) have been developed in different languages in order to ease these text preprocessing tasks.
A Regular Expression is a text string that describes a search pattern which can be used to match or replace patterns inside a string with a minimal amount of code. In this tutorial, we will implement different types of regular expressions in the Python language.
To implement regular expressions, the Python's re package can be used. Import the Python's re package with the following command:
import re
Searching Patterns in a String
One of the most common NLP tasks is to search if a string contains a certain pattern or not. For instance, you may want to perform an operation on the string based on the condition that the string contains a number.
To search a pattern within a string, the match and findall function of the re package is used.
The match Function
Initialize a variable text with a text string as follows:
text = "The film Titanic was released in 1998"
Let's write a regex expression that matches a string of any length and any character:
result = re.match(r".*", text)
The first parameter of the match function is the regex expression that you want to search. Regex expression starts with the alphabet r followed by the pattern that you want to search. The pattern should be enclosed in single or double quotes like any other string.
The above regex expression will match the text string, since we are trying to match a string of any length and any character. If a match is found, the match function returns _sre.SRE_Match object as shown below:
type(result)
Output:
_sre.SRE_Match
Now to find the matched string, you can use the following command:
result.group(0)
Output:
'The film Titanic was released in 1998'
In case if no match is found by the match function, a null object is returned.
Now the previous regex expression matches a string with any length and any character. It will also match an empty string of length zero. To test this, update the value of text variable with an empty string:
text = ""
Now, if you again execute the following regex expression, a match will be found:
result = re.match(r".*", text)
Since we specified to match the string with any length and any character, even an empty string is being matched.
To match a string with a length of at least 1, the following regex expression is used:
result = re.match(r".+", text)
Here the plus sign specifies that the string should have at least one character.
Searching Alphabets
The match function can be used to find any alphabet letters within a string. Let's initialize the text variable with the following text:
text = "The film Titanic was released in 1998"
Now to find all the alphabet letter, both uppercase and lowercase, we can use the following regex expression:
result = re.match(r"[a-zA-z]+", text)
This regex expression states that match the text string for any alphabets from small a to small z or capital A to capital Z. The plus sign specifies that string should have at least one character. Let's print the match found by the above expression:
print(result.group(0))
Output:
The
In the output, you can see that the first word i.e. The is returned. This is because the match function only returns the first match found. In the regex we specified that find the patterns with both small and capital alphabets from a to z. The first match found was The. After the word The there is a space, which is not treated as an alphabet letter, therefore the matching stopped and the expression returned just The, which is the first match.
However, there is a problem with this. If a string starts with a number instead of an alphabet, the match function will return null even if there are alphabets after the number. Let's see this in action:
text = "1998 was the year when the film titanic was released"
result = re.match(r"[a-zA-z]+", text)
type(result)
Output:
NoneType
In the above script, we have updated the text variable and now it starts with a digit. We then used the match function to search for alphabets in the string. Though the text string contains alphabets, null will be returned since match function only matches the first element in the string.
To solve this problem we can use the search function.
The search Function
The search function is similar to the match function i.e. it tries to match the specified pattern. However, unlike the match function, it matches the pattern globally instead of matching only the first element. Therefore, the search function will return a match even if the string doesn't contain an alphabet at the start of the string but contains an alphabet elsewhere in the string, as shown below:
text = "1998 was the year when the film titanic was released"
result = re.search(r"[a-zA-z]+", text)
print(result.group(0))
Output:
was
The search function returns "was" since this is the first match that is found in the text string.
Matching String from the Start
To check if a string starts with a specific word, you can use the carrot key i.e. ^ followed by the word to match with the search function as shown below. Suppose we have the following string:
text = "XYZ 1998 was the year when the film titanic was released"
If we want to find out whether the string starts with "1998", we can use the search function as follows:
result = re.search(r"^1998", text)
type(result)
In the output, null will be returned since the text string doesn't contain "1998" directly at the start.
Now let's change the content text variable and add "1998" at the beginning and then check if "1998" is found at the beginning or not. Execute the following script:
text = "1998 was the year when the film titanic was released"
if re.search(r"^1998", text):
print("Match found")
else:
print("Match not found")
Output:
Match found
Matching Strings from the End
To check whether a string ends with a specific word or not, we can use the word in the regular expression, followed by the dollar sign. The dollar sign marks the end of the statement. Take a look at the following example:
text = "1998 was the year when the film titanic was released"
if re.search(r"1998$", text):
print("Match found")
else:
print("Match not found")
In the above script, we tried to find if the text string ends with "1998", which is not the case.
Output:
Match not found
Now if we update the string and add "1998" at the end of the text string, the above script will return 'Match found' as shown below:
text = "was the year when the film titanic was released 1998"
if re.search(r"1998$", text):
print("Match found")
else:
print("Match not found")
Output:
Match found
Substituting text in a String
Till now we have been using regex to find if a pattern exists in a string. Let's move forward with another advanced regex function i.e. substituting text in a string. The sub function is used for this purpose.
Let's take a simple example of the substitute function. Suppose we have the following string:
text = "The film Pulp Fiction was released in year 1994"
To replace the string "Pulp Fiction" with "Forrest Gump" (another movie released in 1994) we can use the sub function as follows:
result = re.sub(r"Pulp Fiction", "Forrest Gump", text)
The first parameter to the sub function is the regular expression that finds the pattern to substitute. The second parameter is the new text that you want as a replacement for the old text and the third parameter is the text string on which the substitute operation will be performed.
If you print the result variable, you will see the new string.
Now let's substitute all the alphabets in our string with character "X". Execute the following script:
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"[a-z]", "X", text)
print(result)
Output:
TXX XXXX PXXX FXXXXXX XXX XXXXXXXX XX XXXX 1994
It can be seen from the output that all the characters have been replaced except the capital ones. This is because we specified a-z only and not A-Z. There are two ways to solve this problem. You can either specify A-Z in the regular expression along with a-z as follows:
result = re.sub(r"[a-zA-Z]", "X", text)
Or you can pass the additional parameter flags to the sub function and set its value to re.I which refers to case insensitive, as follows:
result = re.sub(r"[a-z]", "X", text, flags=re.I)
More details about different types of flags can be found at Python regex official documentation page.
Shorthand Character Classes
There are different types of shorthand character classes that can be used to perform a variety of different string manipulation functions without having to write complex logic. In this section we will discuss some of them:
Removing Digits from a String
The regex expression to find digits in a string is \d. This pattern can be used to remove digits from a string by replacing them with an empty string of length zero as shown below:
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"\d", "", text)
print(result)
Output:
The film Pulp Fiction was released in year
Removing Alphabet Letters from a String
text = "The film Pulp Fiction was released in year 1994"
result = re.sub(r"[a-z]", "", text, flags=re.I)
print(result)
Output:
1994
Removing Word Characters
If you want to remove all the word characters (letters and numbers) from a string and keep the remaining characters, you can use the \w pattern in your regex and replace it with an empty string of length zero, as shown below:
text = "The film, '@Pulp Fiction' was ? released in % $ year 1994."
result = re.sub(r"\w","", text, flags = re.I)
print(result)
Output:
, '@ ' ? % $ .
The output shows that all the numbers and alphabets have been removed.
Removing Non-Word Characters
To remove all the non-word characters, the \W pattern can be used as follows:
text = "The film, '@Pulp Fiction' was ? released in % $ year 1994."
result = re.sub(r"\W", "", text, flags=re.I)
print(result)
Output:
ThefilmPulpFictionwasreleasedinyear1994
From the output, you can see that everything has been removed (even spaces), except the numbers and alphabets.
Grouping Multiple Patterns
You can group multiple patterns to match or substitute in a string using the square bracket. In fact, we did this when we matched capital and small letters. Let's group multiple punctuation marks and remove them from a string:
text = "The film, '@Pulp Fiction' was ? released _ in % $ year 1994."
result = re.sub(r"[,@\'?\.$%_]", "", text, flags=re.I)
print(result)
Output:
The film Pulp Fiction was released in year 1994
You can see that the string in the text variable had multiple punctuation marks, we grouped all these punctuations in the regex expression using square brackets. It is important to mention that with a dot and a single quote we have to use the escape sequence i.e. backward slash. This is because by default the dot operator is used for any character and the single quote is used to denote a string.
Removing Multiple Spaces
Sometimes, multiple spaces appear between words as a result of removing words or punctuation. For instance, in the output of the last example, there are multiple spaces between in and year. These spaces can be removed using the \s pattern, which refers to a single space.
text = "The film Pulp Fiction was released in year 1994."
result = re.sub(r"\s+"," ", text, flags = re.I)
print(result)
Output:
The film Pulp Fiction was released in year 1994.
In the script above we used the expression \s+ which refers to single or multiple spaces.
Removing Spaces from Start and End
Sometimes we have a sentence that starts or ends with a space, which is often not desirable. The following script removes spaces from the beginning of a sentence:
text = " The film Pulp Fiction was released in year 1994"
result = re.sub(r"^\s+", "", text)
print(result)
Output:
The film Pulp Fiction was released in year 1994
Similarly, to remove space at the end of the string, the following script can be used:
text = "The film Pulp Fiction was released in year 1994 "
result = re.sub(r"\s+$", "", text)
print(result)
Removing a Single Character
Sometimes removing punctuation marks, such as an apostrophe, results in a single character which has no meaning. For instance, if you remove the apostrophe from the word Jacob's and replace it with space, the resultant string is Jacob s. Here the s makes no sense. Such single characters can be removed using regex as shown below:
text = "The film Pulp Fiction s was b released in year 1994"
result = re.sub(r"\s+[a-zA-Z]\s+", " ", text)
print(result)
Output:
The film Pulp Fiction was released in year 1994
The script replaces any small or capital letter between one or more spaces, with a single space.
Splitting a String
String splitting is another very important function. Strings can be split using split function from the re package. The split function returns a list of split tokens. Let's split a string of words where one or more space characters are found, as shown below:
text = "The film Pulp Fiction was released in year 1994 "
result = re.split(r"\s+", text)
print(result)
Output:
['The', 'film', 'Pulp', 'Fiction', 'was', 'released', 'in', 'year', '1994', '']
Similarly, you can use other regex expressions to split a string using the split functions. For instance, the following split function splits string of words when a comma is found:
text = "The film, Pulp Fiction, was released in year 1994"
result = re.split(r"\,", text)
print(result)
Output:
['The film', ' Pulp Fiction', ' was released in year 1994']
Finding All Instances
The match function conducts a match on the first element while the search function conducts a global search on the string and returns the first matched instance.
For instance, if we have the following string:
text = "I want to buy a mobile between 200 and 400 euros"
We want to search all the digits from this string. If we use the search function, only the first occurrence of digits i.e. 200 will be returned as shown below:
result = re.search(r"\d+", text)
print(result.group(0))
Output:
200
On the other hand, the findall function returns a list that contains all the matched utterances as shown below:
text = "I want to buy a mobile between 200 and 400 euros"
result = re.findall(r"\d+", text)
print(result)
Output:
['200', '400']
You can see from the output that both "200" and "400" is returned by the findall function.
Conclusion
In this article we studied some of the most commonly used regex functions in Python. Regular expressions are extremely useful for preprocessing text that can be further used for a variety of applications, such as topic modeling, text classification, sentimental analysis, and text summarization, etc.
17 Aug 2018 3:18pm GMT
Kay Hayen: Nuitka this week #4
Contents
Goto Generators
This continues TWN #3 where I explained what is all about.
Good news is, at the time Python2 generators were largely working with the new ways, in the mean time not only did all of the Python 2.7 test suite pass with goto generators, also did the Python 3.4 test suite, i.e. also the yield from is working with it.
The way it was done is to set m_yieldfrom in generators, and then to enter a state, where the code will only be resumed, when that sub-generator that currently it is yielding from, is finished. That makes it very much like normal yield. In fact, code generation is hardly different there.
Since the whole purpose is to get rid of make/get/setcontext, the next stop is coroutines. They have async for, async with and await but at the end of the day, the implementation comes down to yield from really with only a lot of sugar applied.
Right now, I am debugging "goto coroutines". It's hard to tell when it will be finished, and then asyncgen will be waiting still.
This is easily the largest change in a long time, esp. due to the heap storage changes that I already discussed. One this is finished, I expect to turn towards C types with relative ease.
Tox Plugin
Anthony Shaw took on Tox and Nuitka and created a plugin that allows using Nuitka. I am still wrapping my head around these things. It's only a proof of concept yet. I will give it more coverage in the future.
Hotfixes
So there have even more hotfixes. One addresses memory leaks found with the yield from while I was adding tests. Usually if I encounter an old issue that has a small fix, that is what I do, push out a hotfix using the git flow model. Also nested namespace packages for Python3, those are the ones without a __init__.py were not working after the original directory was removed, and that got fixed.
And right now, I have hotfixes for frames close method, which apparently was never updated to work properly for coroutines and asyncgen. That is going to be in the next hotfix.
Plans
So the heap storage seems pretty complete now, and goto generators are on the final stretch. As always, things feel right around the corner. But it's unclear how much longer I will have to debug. I am pretty sure the bare work of doing asyncgen is going to be low. Debugging that too then, that is the hard part.
A new release seems justified, but I kind of do not want to make it without that major new code used. Because apparently during the debugging, I tend to find issues that need hotfixes, so I will wait for the goto generator work to finish.
17 Aug 2018 12:55pm GMT
Kay Hayen: Nuitka this week #4
Contents
Goto Generators
This continues TWN #3 where I explained what is all about.
Good news is, at the time Python2 generators were largely working with the new ways, in the mean time not only did all of the Python 2.7 test suite pass with goto generators, also did the Python 3.4 test suite, i.e. also the yield from is working with it.
The way it was done is to set m_yieldfrom in generators, and then to enter a state, where the code will only be resumed, when that sub-generator that currently it is yielding from, is finished. That makes it very much like normal yield. In fact, code generation is hardly different there.
Since the whole purpose is to get rid of make/get/setcontext, the next stop is coroutines. They have async for, async with and await but at the end of the day, the implementation comes down to yield from really with only a lot of sugar applied.
Right now, I am debugging "goto coroutines". It's hard to tell when it will be finished, and then asyncgen will be waiting still.
This is easily the largest change in a long time, esp. due to the heap storage changes that I already discussed. One this is finished, I expect to turn towards C types with relative ease.
Tox Plugin
Anthony Shaw took on Tox and Nuitka and created a plugin that allows using Nuitka. I am still wrapping my head around these things. It's only a proof of concept yet. I will give it more coverage in the future.
Hotfixes
So there have even more hotfixes. One addresses memory leaks found with the yield from while I was adding tests. Usually if I encounter an old issue that has a small fix, that is what I do, push out a hotfix using the git flow model. Also nested namespace packages for Python3, those are the ones without a __init__.py were not working after the original directory was removed, and that got fixed.
And right now, I have hotfixes for frames close method, which apparently was never updated to work properly for coroutines and asyncgen. That is going to be in the next hotfix.
Plans
So the heap storage seems pretty complete now, and goto generators are on the final stretch. As always, things feel right around the corner. But it's unclear how much longer I will have to debug. I am pretty sure the bare work of doing asyncgen is going to be low. Debugging that too then, that is the hard part.
A new release seems justified, but I kind of do not want to make it without that major new code used. Because apparently during the debugging, I tend to find issues that need hotfixes, so I will wait for the goto generator work to finish.
17 Aug 2018 12:55pm GMT
PyBites: A Python Orientation - How to Get Started
Python is a wonderful language for both beginners and expert programmers, but getting started can be daunting. Which version should I use? Which editors are best? What do you mean there are different implementations and environments? Here's a guide to help navigate these big FAQs.
tl;dr
For most people:
- Use the latest version of Python 3.
- Use the CPython implementation.
- Use pipenv to manage packages and installations.
- Use Visual Studio Code or PyCharm for editing code.
Which Version?
Python 2 and Python 3 are actually different languages. The differences go deeper than just print statements. The What's New in Python page on the official doc site lists all the gory details, and decent articles showcasing differences can be found here, here, and here. Although Python 3 is newer, Python 2 remains prevalent. Most popular packages use Python packaging tools to support both versions. The Python Wiki makes it clear that Python 3 is the better choice:
Python 2.x is legacy, Python 3.x is the present and future of the language
Furthermore, Python 2 will reach end-of-life in 2020. The Python team will continue to provide bug fixes for 2.7 until 2020 (PEP 373), but there will be no new language features and no 2.8 (PEP 404). (Originally, end-of-life was planned for 2015, but it was pushed back by 5 years.) There is even a Python 2.7 Countdown clock online.
Which Implementation?
In purest terms, "Python" is a language specification. An implementation provides the language processing tools (compiler, interpreter, etc.) to run Python programs. The Hitchhiker's Guide to Python has a great article entitled Picking an Interpreter that provides a good summary of available interpreters. Others are listed on python.org and the Python Wiki. The table below provides a quick overview of the big ones.
- most widely used implementation
- the reference implementation
- has the most libraries and support
- implemented in C
- supports Python 2 and 3
- much faster than CPython
- much more memory efficient
- implemented in RPython
- supports Python 2 and 3
- implemented in Java
- runs on the JVM
- supports Python 2
- only a sandbox for Python 3
- no project updates since May 2015
- implemented for .NET
- lets Python libs call .NET and vice versa
- supports Python 2
- integrates CPython with .NET/Mono runtime
- supports Python 2 and 3
- branch of CPython with real threading
- optimized for microcontrollers
- uses a subset of the standard library
Unless you have a very specific reason, just use CPython. In fact, most people are referring to CPython when they say "Python." CPython has the most compatibility, the widest package library, and the richest support. If you really need speed, consider PyPy.
Managing Installations
pip is the standard tool for installing Python packages. The simplest way to install Python is to install it "globally" for the system. In fact, some operating systems like macOS and Ubuntu have Python pre-installed. However, global installation has limitations:
- You may want to develop packages for both versions 2 and 3.
- You may not have permissions to add new packages globally.
- Different projects may require different versions of packages.
These problems can be solved by using "virtual" environments. A virtual environment is like a local Python installation with a specific package set. For example, I have created virtual environments for Python as part of Jenkins build jobs, since I did not have permission to install special automation packages globally on the Jenkins slaves.
The standard virtual environment tool for Python is venv, which has been packaged with (C)Python since 3.3. (venv had a command line wrapper named pyvenv, but this was deprecated in 3.6.) Another older but still popular third-party tool is virtualenv. As explained in this Reddit post, venv _is the Python-sanctioned replacement for _virtualenv. However, virtualenv supports Python 2, whereas venv does not. Conda is an environment manager popular with the science and data communities, and it can support other languages in addition to Python.
That being said, there is a relatively new package manager taking the Python world by storm: pipenv. Pipenv combines pip, Pipfile, and virtualenv into an easy workflow with simple commands. Personally, I find it to be very helpful. However, it has caused some controversy (see Reddit), and it may not be applicable for all scenarios (see Chris Warrick's article). My recommendation is to use pipenv for new projects if it meets your needs.
Editors and IDEs
After setting up your Python environment, you are ready to start programming! There are two routes to take for text editing: source code editors and integrated development environments.
Source code editors are lightweight but often include basics like syntax highlighting and basic auto-completion. They're great for quick edits and light scripting. Many have plugins. Popular choices are Visual Studio Code, Sublime, Atom, and Notepad++. My current favorite is Visual Studio Code because the Python extensions are stellar and settings are simple - just remember to install the extensions you need! I use it personally for Django development.
For more intense development, I highly recommend an IDE like JetBrains PyCharm, PyDev for Eclipse, Wing Python IDE, or Eric. IDEs provide rich development support, especially for larger apps that use frameworks like Django, Pyramid, and SQLAlchemy. They also make testing easier with plugins for test frameworks like pytest, behave, and others. PyCharm and PyDev are particularly nice because they can integrate into their larger IDEs (IntelliJ IDEA and Eclipse, respectively) to handle more languages. Personally, I prefer PyCharm, but advanced features require a paid license.
Pythonese
The Python community throws around a few terms you should know:
- describes idiomatic code for Python
- closely related to conciseness, readability, and elegance
- highly recommended to use
- follow style guidelines
- someone who loves the Python language
- often an advanced Python programmer
- a programmer who uses Python to solve problems
- the list of guiding principles for Python's design
- run "import this" to see them
The Python Software Foundation (PSF)
- non-profit org
- keeps Python going strong
- support them!
- the annual Python conference held in North America
- GO - it will change your life!
- several other conferences are held worldwide
Benevolent Dictator for Life(BDFL)
- Guido van Rossum
- the inventor of Python
- resigned in July 2018 but remains BDFL Emeritus
This article was originally posted at AutomationPanda.com and has been reposted here with permission.
Keep Calm and Code in Python!
17 Aug 2018 7:41am GMT
PyBites: A Python Orientation - How to Get Started
Python is a wonderful language for both beginners and expert programmers, but getting started can be daunting. Which version should I use? Which editors are best? What do you mean there are different implementations and environments? Here's a guide to help navigate these big FAQs.
tl;dr
For most people:
- Use the latest version of Python 3.
- Use the CPython implementation.
- Use pipenv to manage packages and installations.
- Use Visual Studio Code or PyCharm for editing code.
Which Version?
Python 2 and Python 3 are actually different languages. The differences go deeper than just print statements. The What's New in Python page on the official doc site lists all the gory details, and decent articles showcasing differences can be found here, here, and here. Although Python 3 is newer, Python 2 remains prevalent. Most popular packages use Python packaging tools to support both versions. The Python Wiki makes it clear that Python 3 is the better choice:
Python 2.x is legacy, Python 3.x is the present and future of the language
Furthermore, Python 2 will reach end-of-life in 2020. The Python team will continue to provide bug fixes for 2.7 until 2020 (PEP 373), but there will be no new language features and no 2.8 (PEP 404). (Originally, end-of-life was planned for 2015, but it was pushed back by 5 years.) There is even a Python 2.7 Countdown clock online.
Which Implementation?
In purest terms, "Python" is a language specification. An implementation provides the language processing tools (compiler, interpreter, etc.) to run Python programs. The Hitchhiker's Guide to Python has a great article entitled Picking an Interpreter that provides a good summary of available interpreters. Others are listed on python.org and the Python Wiki. The table below provides a quick overview of the big ones.
- most widely used implementation
- the reference implementation
- has the most libraries and support
- implemented in C
- supports Python 2 and 3
- much faster than CPython
- much more memory efficient
- implemented in RPython
- supports Python 2 and 3
- implemented in Java
- runs on the JVM
- supports Python 2
- only a sandbox for Python 3
- no project updates since May 2015
- implemented for .NET
- lets Python libs call .NET and vice versa
- supports Python 2
- integrates CPython with .NET/Mono runtime
- supports Python 2 and 3
- branch of CPython with real threading
- optimized for microcontrollers
- uses a subset of the standard library
Unless you have a very specific reason, just use CPython. In fact, most people are referring to CPython when they say "Python." CPython has the most compatibility, the widest package library, and the richest support. If you really need speed, consider PyPy.
Managing Installations
pip is the standard tool for installing Python packages. The simplest way to install Python is to install it "globally" for the system. In fact, some operating systems like macOS and Ubuntu have Python pre-installed. However, global installation has limitations:
- You may want to develop packages for both versions 2 and 3.
- You may not have permissions to add new packages globally.
- Different projects may require different versions of packages.
These problems can be solved by using "virtual" environments. A virtual environment is like a local Python installation with a specific package set. For example, I have created virtual environments for Python as part of Jenkins build jobs, since I did not have permission to install special automation packages globally on the Jenkins slaves.
The standard virtual environment tool for Python is venv, which has been packaged with (C)Python since 3.3. (venv had a command line wrapper named pyvenv, but this was deprecated in 3.6.) Another older but still popular third-party tool is virtualenv. As explained in this Reddit post, venv _is the Python-sanctioned replacement for _virtualenv. However, virtualenv supports Python 2, whereas venv does not. Conda is an environment manager popular with the science and data communities, and it can support other languages in addition to Python.
That being said, there is a relatively new package manager taking the Python world by storm: pipenv. Pipenv combines pip, Pipfile, and virtualenv into an easy workflow with simple commands. Personally, I find it to be very helpful. However, it has caused some controversy (see Reddit), and it may not be applicable for all scenarios (see Chris Warrick's article). My recommendation is to use pipenv for new projects if it meets your needs.
Editors and IDEs
After setting up your Python environment, you are ready to start programming! There are two routes to take for text editing: source code editors and integrated development environments.
Source code editors are lightweight but often include basics like syntax highlighting and basic auto-completion. They're great for quick edits and light scripting. Many have plugins. Popular choices are Visual Studio Code, Sublime, Atom, and Notepad++. My current favorite is Visual Studio Code because the Python extensions are stellar and settings are simple - just remember to install the extensions you need! I use it personally for Django development.
For more intense development, I highly recommend an IDE like JetBrains PyCharm, PyDev for Eclipse, Wing Python IDE, or Eric. IDEs provide rich development support, especially for larger apps that use frameworks like Django, Pyramid, and SQLAlchemy. They also make testing easier with plugins for test frameworks like pytest, behave, and others. PyCharm and PyDev are particularly nice because they can integrate into their larger IDEs (IntelliJ IDEA and Eclipse, respectively) to handle more languages. Personally, I prefer PyCharm, but advanced features require a paid license.
Pythonese
The Python community throws around a few terms you should know:
- describes idiomatic code for Python
- closely related to conciseness, readability, and elegance
- highly recommended to use
- follow style guidelines
- someone who loves the Python language
- often an advanced Python programmer
- a programmer who uses Python to solve problems
- the list of guiding principles for Python's design
- run "import this" to see them
The Python Software Foundation (PSF)
- non-profit org
- keeps Python going strong
- support them!
- the annual Python conference held in North America
- GO - it will change your life!
- several other conferences are held worldwide
Benevolent Dictator for Life(BDFL)
- Guido van Rossum
- the inventor of Python
- resigned in July 2018 but remains BDFL Emeritus
This article was originally posted at AutomationPanda.com and has been reposted here with permission.
Keep Calm and Code in Python!
17 Aug 2018 7:41am GMT
10 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: King Willams Town Bahnhof
Gestern musste ich morgens zur Station nach KWT um unsere Rerservierten Bustickets für die Weihnachtsferien in Capetown abzuholen. Der Bahnhof selber ist seit Dezember aus kostengründen ohne Zugverbindung - aber Translux und co - die langdistanzbusse haben dort ihre Büros.
Größere Kartenansicht
10 Nov 2011 10:57am GMT
09 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein
Niemand ist besorgt um so was - mit dem Auto fährt man einfach durch, und in der City - nahe Gnobie- "ne das ist erst gefährlich wenn die Feuerwehr da ist" - 30min später auf dem Rückweg war die Feuerwehr da.
09 Nov 2011 8:25pm GMT
08 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Brai Party
Brai = Grillabend o.ä.
Die möchte gern Techniker beim Flicken ihrer SpeakOn / Klinke Stecker Verzweigungen...
Die Damen "Mamas" der Siedlung bei der offiziellen Eröffnungsrede
Auch wenn weniger Leute da waren als erwartet, Laute Musik und viele Leute ...
Und natürlich ein Feuer mit echtem Holz zum Grillen.
08 Nov 2011 2:30pm GMT
07 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Lumanyano Primary
One of our missions was bringing Katja's Linux Server back to her room. While doing that we saw her new decoration.
Björn, Simphiwe carried the PC to Katja's school
07 Nov 2011 2:00pm GMT
06 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Nelisa Haircut
Today I went with Björn to Needs Camp to Visit Katja's guest family for a special Party. First of all we visited some friends of Nelisa - yeah the one I'm working with in Quigney - Katja's guest fathers sister - who did her a haircut.
African Women usually get their hair done by arranging extensions and not like Europeans just cutting some hair.
In between she looked like this...
And then she was done - looks amazing considering the amount of hair she had last week - doesn't it ?
06 Nov 2011 7:45pm GMT
05 Nov 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Mein Samstag
Irgendwie viel mir heute auf das ich meine Blogposts mal ein bischen umstrukturieren muss - wenn ich immer nur von neuen Plätzen berichte, dann müsste ich ja eine Rundreise machen. Hier also mal ein paar Sachen aus meinem heutigen Alltag.
Erst einmal vorweg, Samstag zählt zumindest für uns Voluntäre zu den freien Tagen.
Dieses Wochenende sind nur Rommel und ich auf der Farm - Katja und Björn sind ja mittlerweile in ihren Einsatzstellen, und meine Mitbewohner Kyle und Jonathan sind zu Hause in Grahamstown - sowie auch Sipho der in Dimbaza wohnt.
Robin, die Frau von Rommel ist in Woodie Cape - schon seit Donnerstag um da ein paar Sachen zur erledigen.
Naja wie dem auch sei heute morgen haben wir uns erstmal ein gemeinsames Weetbix/Müsli Frühstück gegönnt und haben uns dann auf den Weg nach East London gemacht. 2 Sachen waren auf der Checkliste Vodacom, Ethienne (Imobilienmakler) außerdem auf dem Rückweg die fehlenden Dinge nach NeedsCamp bringen.
Nachdem wir gerade auf der Dirtroad losgefahren sind mussten wir feststellen das wir die Sachen für Needscamp und Ethienne nicht eingepackt hatten aber die Pumpe für die Wasserversorgung im Auto hatten.
Also sind wir in EastLondon ersteinmal nach Farmerama - nein nicht das onlinespiel farmville - sondern einen Laden mit ganz vielen Sachen für eine Farm - in Berea einem nördlichen Stadteil gefahren.
In Farmerama haben wir uns dann beraten lassen für einen Schnellverschluss der uns das leben mit der Pumpe leichter machen soll und außerdem eine leichtere Pumpe zur Reperatur gebracht, damit es nicht immer so ein großer Aufwand ist, wenn mal wieder das Wasser ausgegangen ist.
Fego Caffé ist in der Hemmingways Mall, dort mussten wir und PIN und PUK einer unserer Datensimcards geben lassen, da bei der PIN Abfrage leider ein zahlendreher unterlaufen ist. Naja auf jeden Fall speichern die Shops in Südafrika so sensible Daten wie eine PUK - die im Prinzip zugang zu einem gesperrten Phone verschafft.
Im Cafe hat Rommel dann ein paar online Transaktionen mit dem 3G Modem durchgeführt, welches ja jetzt wieder funktionierte - und übrigens mittlerweile in Ubuntu meinem Linuxsystem perfekt klappt.
Nebenbei bin ich nach 8ta gegangen um dort etwas über deren neue Deals zu erfahren, da wir in einigen von Hilltops Centern Internet anbieten wollen. Das Bild zeigt die Abdeckung UMTS in NeedsCamp Katjas Ort. 8ta ist ein neuer Telefonanbieter von Telkom, nachdem Vodafone sich Telkoms anteile an Vodacom gekauft hat müssen die komplett neu aufbauen.
Wir haben uns dazu entschieden mal eine kostenlose Prepaidkarte zu testen zu organisieren, denn wer weis wie genau die Karte oben ist ... Bevor man einen noch so billigen Deal für 24 Monate signed sollte man wissen obs geht.
Danach gings nach Checkers in Vincent, gesucht wurden zwei Hotplates für WoodyCape - R 129.00 eine - also ca. 12€ für eine zweigeteilte Kochplatte.
Wie man sieht im Hintergrund gibts schon Weihnachtsdeko - Anfang November und das in Südafrika bei sonnig warmen min- 25°C
Mittagessen haben wir uns bei einem Pakistanischen Curry Imbiss gegönnt - sehr empfehlenswert !
Naja und nachdem wir dann vor ner Stunde oder so zurück gekommen sind habe ich noch den Kühlschrank geputzt den ich heute morgen zum defrosten einfach nach draußen gestellt hatte. Jetzt ist der auch mal wieder sauber und ohne 3m dicke Eisschicht...
Morgen ... ja darüber werde ich gesondert berichten ... aber vermutlich erst am Montag, denn dann bin ich nochmal wieder in Quigney(East London) und habe kostenloses Internet.
05 Nov 2011 4:33pm GMT
31 Oct 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Sterkspruit Computer Center
Sterkspruit is one of Hilltops Computer Centres in the far north of Eastern Cape. On the trip to J'burg we've used the opportunity to take a look at the centre.
 |
| Pupils in the big classroom |
 |
| The Trainer |
 |
| School in Countryside |
 |
| Adult Class in the Afternoon |
 |
| "Town" |
31 Oct 2011 4:58pm GMT
Benedict Stein: Technical Issues
What are you doing in an internet cafe if your ADSL and Faxline has been discontinued before months end. Well my idea was sitting outside and eating some ice cream.
At least it's sunny and not as rainy as on the weekend.
31 Oct 2011 3:11pm GMT
30 Oct 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Nellis Restaurant
For those who are traveling through Zastron - there is a very nice Restaurant which is serving delicious food at reasanable prices.
In addition they're selling home made juices jams and honey.
 |
| interior |
 |
| home made specialities - the shop in the shop |
 |
| the Bar |
30 Oct 2011 4:47pm GMT
29 Oct 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: The way back from J'burg
Having the 10 - 12h trip from J'burg back to ELS I was able to take a lot of pcitures including these different roadsides
 |
| Plain Street |
 |
| Orange River in its beginngings (near Lesotho) |
 |
| Zastron Anglican Church |
 |
| The Bridge in Between "Free State" and Eastern Cape next to Zastron |
 |
| my new Background ;) |
 |
| If you listen to GoogleMaps you'll end up traveling 50km of gravel road - as it was just renewed we didn't have that many problems and saved 1h compared to going the official way with all it's constructions sites |
 |
| Freeway |
 |
| getting dark |
29 Oct 2011 4:23pm GMT
28 Oct 2011
Python Software Foundation | GSoC'11 Students
Benedict Stein: Wie funktioniert eigentlich eine Baustelle ?
Klar einiges mag anders sein, vieles aber gleich - aber ein in Deutschland täglich übliches Bild einer Straßenbaustelle - wie läuft das eigentlich in Südafrika ?
Ersteinmal vorweg - NEIN keine Ureinwohner die mit den Händen graben - auch wenn hier mehr Manpower genutzt wird - sind sie fleißig mit Technologie am arbeiten.
 |
| Eine ganz normale "Bundesstraße" |
 |
| und wie sie erweitert wird |
 |
| gaaaanz viele LKWs |
 |
| denn hier wird eine Seite über einen langen Abschnitt komplett gesperrt, so das eine Ampelschaltung mit hier 45 Minuten Wartezeit entsteht |
 |
||
| Aber wenigstens scheinen die ihren Spaß zu haben ;) - Wie auch wir denn gücklicher Weise mussten wir nie länger als 10 min. warten. |
28 Oct 2011 4:20pm GMT