28 May 2020

feedPlanet Python

Matt Layman: Bread and Butter Django - Building SaaS #58

In this episode, I worked on a views and templates. There are a number of core pages that are required to flesh out the minimal interface for the app. We're building them. I began by showing the page that we were going to work on. I outlined the changes I planned to make, then we started. The first thing we added was data about the school year, the main model on display in the page.

28 May 2020 12:00am GMT

Matt Layman: Bread and Butter Django - Building SaaS #58

In this episode, I worked on a views and templates. There are a number of core pages that are required to flesh out the minimal interface for the app. We're building them. I began by showing the page that we were going to work on. I outlined the changes I planned to make, then we started. The first thing we added was data about the school year, the main model on display in the page.

28 May 2020 12:00am GMT

27 May 2020

feedPlanet Python

Gal Varoquaux: Technical discussions are hard; a few tips

Note

This post discuss the difficulties of communicating while developing open-source projects and tries to gives some simple advice.

A large software project is above all a social exercise in which technical experts try to reach good decisions together, for instance on github pull requests. But communication is difficult, in particular between diverging points of view. It is easy to underestimate how much well-intended persons can misunderstand each-other and get hurt, in open source as elsewhere. Knowing why there are communication challenges can help, as well as applying a few simple rules.

Contents


The first challenge is to understand the other's point of view: the different parties see the problem differently.

Maintainer's anxiety

Open source can be anxiety-generating for the maintainers

Maintainers ensure the quality and the long-term life of an open-source project. As such, they feel responsible for any shortcoming in the product. In addition, they often do this work because they care, even though it may not bring any financial support. But they can quickly become a converging point of anxiety-generating feedback:

  • Code has bugs; the more code, the more bugs. Watching a issue tracker fill up with a long list of bugs is frightening to people who feel in charge.
  • Given that maintainers are visible and qualified, they become the target of constant requests for attention: from pleas to prioritize a specific issue to solicitations for advice.
  • A small fraction of these interactions come as plain aggressions. I have been insulted many times by unsatisfied users. Each time, it hurts me a lot. My policy is to disengage from the conversation, but I am left shaking and staring at my computer in the evening.

Related writings

Ralf Gommers discusses the cost of an open source contribution, from the point of view of the maintainer.

Ilya Grigorik suggests: Don't push your pull request.

Brett Cannon: Setting expectations for open source participation

The more popular a project, the more weight it puts on its maintainers' shoulders. A consequence is that maintainers are tired, and can sometimes approach discussions in a defensive way. Also, we may be plain scared of integrating a code that we do not fully comprehend.

Open-source developers may even, unconsciously, adopt a simple, but unfortunate, protection mechanism: being rude. The logic is flawless: if I am nasty to people, or I set unreasonnable expectations, people will let me alone. Alas, this strategy leads to toxic environments. It not only makes people unhappy but also harms the community dynamics that ground the excellence of open source.

The danger abusive gatekeeping

A maintainer quickly learns that every piece of code, no matter how cute it might be, will give him work in the long run, just like a puppy. This is unavoidable given that the complexity of code grows faster than its number of features [1], and, even for a company as rich as Google, project maintenance becomes intractable on huge projects [2].

[1] An Experiment on Unit Increase in Problem Complexity, Woodfield 1979
[2] To quote tensorflow developers "Every [code addition] takes around 16 CPU/GPU hours of [quality control]. As such, we cannot just run every [code addition] through the [quality control] infrastructure."

A maintainer's job is to say no often, to protect the project. But, as any gatekeeping, it can unfortunately become an excercise in unchecked power. Making objective choices for these difficult decisions is hard, and we all tend naturally to trust more people that we know.


Most often we are not aware of our shortcomings, let alone are we doing them on purpose.

Contributor's fatigue

A new contributor starting a conversation with a group of seasoned project maintainers may easily feel an imposter. The new contributor knows less about the project. In addition, he or she is engaging with a group of people that know each-other well, and is not yet part of that inner group.

This person does not know the code base, or the conventions, and must make extra efforts, compared to the seasoned developers, to propose a contribution suitable for the project. Often, he or she does not understand fully the reasons for the project guidelines, or for the feedback given. Request for changes can easily be seen as trifles.

Integrating the contribution can often be a lengthy process -in particular in scikit-learn. Indeed, it will involve not only shaping up the contribution, but also learning the skills and discovering the process. These long cycles can undermine motivation: humans need successes to feel enthusiasm. Also, the contributor may legitimately worry: Will all these efforts be fruitful? Will the contribution make its way to the project?

Note that for these reasons, it is recommended to start contributing with very simple features, and to seek feedback on the scope of the contribution before writing the code.

Finally, contributors are seldom paid to work on the project, and there is no single line of command that makes decisions and controls incentives for all the people on the project. No one is responsible when things go astray, which means that the weight falls on the shoulder of the individuals.


The danger behind the lengthy cycle of reviews and improvements needed to contribute is death by a thousands cuts. The contributor looses motivation, and no longer finds the energy to finish the work.

How about users?

This article is focused on developers. Yet, users are also an important part of the discussion around open source.

Often communication failures with users are due to frustration. Frustration of being unable to use the software, of hitting a bug, of seeing an important issue still not addressed. This frustration stems from incorrect expectations, which can often be traced to misunderstanding of the processes and the dynamics. Managing expectations is important to improve the dialogue, via the documentation, via notes on the issue tracker.


Communication is hard

Communication is hard: messages are sometimes received differently than we would like. Overworked people discussing very technically challenging issues only makes the matter worse. I have seen people not come across well, while I know they are absolutely lovely and caring.

We are human beings; we are limited; we misunderstand things, and we have feelings.


Emotions - My most vivid memory of a communication failure was when I was a sailing instructor. Trainees that were under my responsibility had put themselves at risk, causing me a lot of worry. During the debrief, I was angry. My failure to convey the messages without emotional loading undermined my leadership on the group, putting everybody at risk for the rest of the week.

Inability to understand the others' point of view, or to communicate ours, can bring in emotions. Emotions most often impedes technical communication.


Limited attention - We, in particular maintainers, are bombarded with email, notifications, text and code to read. As a consequence, it is easy to read things too fast, to stop in the middle, to forget.


Language barriers - Most discussions happen in English; but most of us are not native English speakers. We may hide well our difficulties, but nuances are often lost.


Clique effects - Most interactions in open source are done in writing, with low communication bandwidth. It can be much harder to convince a maintainer on the other side of the world than a colleague in the same room. Schools of thoughts naturally emerge when people work a lot together. These create bubbles, where we have the impression that everything we say is obvious and uncontroversial, and yet we fail to convince people outside of our bubble.


Little things that help

Communication can improved by continuously working on it [3]. It may be obvious to some, but it personally took me many years to learn.

[3] Training materials for managers often discuss communication, and give tricks. I am sure that there are better references than my list below. But that's the best I can do.

Hear the other: exchange

Related presentation

How can we have healthier technical discussions? by Nathaniel J. Smith

Foster multiway discussions - The goal of a technical discussion is to come up to the best solution. Better solutions emerge via confronting different points of view: a single brilliant individual probably cannot find or recognize the best solution alone.

  • Integrate input from as many perspectives as possible.
  • Make sure everyone feels heard.

Don't seek victory - Most important to keep in mind is that giving up on an argument and accepting the other point of view is a perfectly valid option. I naturally biased to think that my view on topics dear to me is the right one. However, I've learned that adopting the view of the other could bring a lot to the social dynamics of a project: we are often debating over details and the bigger benefit comes from moving forward.

In addition, if several very bright people have different conclusions than me about something that they've thought a lot, who am I to disagree?


Convey ideas well: pedagogy

Explain - Give the premises of your thoughts. Unroll your thought processes. People are not sitting in your head, and need to hear not only your conclusion, but how you got there.

Repeat things - Account for the fact that people can forget, and never hesitate to gently restate important points. Reformulating differently can also help explaining.

Keep it short - A typical reading speed is around 200 words a minute. People have limited time and attention span. The greatest help you can provide to your reader is to condense your ideas: let us avoid long threads that require several dozens of minutes to read and digest. There is a tension between this point and the above. My suggestion: remove every word that is not useful, move details to footnotes or postscriptums.


Cater for emotions: tone

Related good advice

Mozilla participation guide, expected behavior section

Stay technical - Always try to get to the technical aspect of the matter, and never the human. Give specific code and wording suggestions. When explaining a decision, give technical arguments, even if they feel obvious to you.

Be positive - Being positive in general helps people feeling happy and motivated. It is well known that positive feedback leads to quicker progress than negative, as revealed eg by studies of class rooms. I am particularly guilty of this: I always forget to say something nice, although I may be super impressed by a contribution. Likewise, avoid negative words when giving feedback (stay technical).

Avoid "you" - The mere use of the pronoun "you" puts the person we are talking to in the center of message. But the message should not be about the person, it should be about the work. It's very easy to react emotionally when it's about us. The passive voice can be useful to avoid putting people as the topic. If the topic is indeed people, sometimes "we" is an adequate substitute for "you".

Assume good faith - There are so many misunderstandings that can happen. People forget things, people make mistakes, people fail to convey their messages. Most often, all these failures are in good faith, and misunderstandings are legitimate. In the rare cases there might possibly be some bad faith, accounting for it will only make communication worse, not better. Along the same line, we should ignore when we feel assaulted or insulted, and avoid replying in kind.

Choose words wisely - The choice of words matter, because they convey implicit messages. In particular, avoid terms that carry judgement values: "good" or "bad". For example "This is done wrong" (note that this sentence already avoids "you"), could be replaced by "There might be more numerically stable / efficient way of doing it" (note also the use of precise technical wording rather than the generic term "better").

Use moderating words - Try to leave room for the other in the discussion. Statements too assertive close the door to different points of view: "this must be changed" (note the lack of "you") should be avoided while "this should be changed" is better. For this reason, this article is riddled with words such as "tend", "often", "feel", "may", "might".

Don't blame someone else - If you feel that there is some pattern that you would like to change, do not point fingers, do not blame others. Rather, point yourself at the center of the story, find an example of this pattern with you, and the message should be that "it is a pattern that we should avoid. "We" is such a powerful term. It unites; it builds a team.

Give your understanding - If you feel that there is a misunderstanding, explain how you are feeling. But do it using "I", and not "you", and acknowledge the subjectivity: "I feel ignored" rather than "you are ignoring me". Even better: only talk about the feeling: "I am loosing motivation, because this is not moving forward", or "I think that am failing to convey why this numerical problem is such an important issue" (note the use of "I think", which avoids casting the situation as necessarily true).

Implicit messages

The four sides view of communication highlights the multiple messages present even in simple statements.


I hope this can be useful. I personally try to apply these rules, because I want to work better with others.

Thanks

to many who gave me feedback: Adrin Jalali, Andreas Mueller, Elizabeth DuPre, Emmanuelle Gouillart, Guillaume Lemaitre, Joel Nothman, Joris Van den Bossche, Nicolas Hug.



PS: note how many times I've used "you" above. I can clearly get better at communication!

27 May 2020 10:00pm GMT

Gal Varoquaux: Technical discussions are hard; a few tips

Note

This post discuss the difficulties of communicating while developing open-source projects and tries to gives some simple advice.

A large software project is above all a social exercise in which technical experts try to reach good decisions together, for instance on github pull requests. But communication is difficult, in particular between diverging points of view. It is easy to underestimate how much well-intended persons can misunderstand each-other and get hurt, in open source as elsewhere. Knowing why there are communication challenges can help, as well as applying a few simple rules.

Contents


The first challenge is to understand the other's point of view: the different parties see the problem differently.

Maintainer's anxiety

Open source can be anxiety-generating for the maintainers

Maintainers ensure the quality and the long-term life of an open-source project. As such, they feel responsible for any shortcoming in the product. In addition, they often do this work because they care, even though it may not bring any financial support. But they can quickly become a converging point of anxiety-generating feedback:

  • Code has bugs; the more code, the more bugs. Watching a issue tracker fill up with a long list of bugs is frightening to people who feel in charge.
  • Given that maintainers are visible and qualified, they become the target of constant requests for attention: from pleas to prioritize a specific issue to solicitations for advice.
  • A small fraction of these interactions come as plain aggressions. I have been insulted many times by unsatisfied users. Each time, it hurts me a lot. My policy is to disengage from the conversation, but I am left shaking and staring at my computer in the evening.

Related writings

Ralf Gommers discusses the cost of an open source contribution, from the point of view of the maintainer.

Ilya Grigorik suggests: Don't push your pull request.

Brett Cannon: Setting expectations for open source participation

The more popular a project, the more weight it puts on its maintainers' shoulders. A consequence is that maintainers are tired, and can sometimes approach discussions in a defensive way. Also, we may be plain scared of integrating a code that we do not fully comprehend.

Open-source developers may even, unconsciously, adopt a simple, but unfortunate, protection mechanism: being rude. The logic is flawless: if I am nasty to people, or I set unreasonnable expectations, people will let me alone. Alas, this strategy leads to toxic environments. It not only makes people unhappy but also harms the community dynamics that ground the excellence of open source.

The danger abusive gatekeeping

A maintainer quickly learns that every piece of code, no matter how cute it might be, will give him work in the long run, just like a puppy. This is unavoidable given that the complexity of code grows faster than its number of features [1], and, even for a company as rich as Google, project maintenance becomes intractable on huge projects [2].

[1] An Experiment on Unit Increase in Problem Complexity, Woodfield 1979
[2] To quote tensorflow developers "Every [code addition] takes around 16 CPU/GPU hours of [quality control]. As such, we cannot just run every [code addition] through the [quality control] infrastructure."

A maintainer's job is to say no often, to protect the project. But, as any gatekeeping, it can unfortunately become an excercise in unchecked power. Making objective choices for these difficult decisions is hard, and we all tend naturally to trust more people that we know.


Most often we are not aware of our shortcomings, let alone are we doing them on purpose.

Contributor's fatigue

A new contributor starting a conversation with a group of seasoned project maintainers may easily feel an imposter. The new contributor knows less about the project. In addition, he or she is engaging with a group of people that know each-other well, and is not yet part of that inner group.

This person does not know the code base, or the conventions, and must make extra efforts, compared to the seasoned developers, to propose a contribution suitable for the project. Often, he or she does not understand fully the reasons for the project guidelines, or for the feedback given. Request for changes can easily be seen as trifles.

Integrating the contribution can often be a lengthy process -in particular in scikit-learn. Indeed, it will involve not only shaping up the contribution, but also learning the skills and discovering the process. These long cycles can undermine motivation: humans need successes to feel enthusiasm. Also, the contributor may legitimately worry: Will all these efforts be fruitful? Will the contribution make its way to the project?

Note that for these reasons, it is recommended to start contributing with very simple features, and to seek feedback on the scope of the contribution before writing the code.

Finally, contributors are seldom paid to work on the project, and there is no single line of command that makes decisions and controls incentives for all the people on the project. No one is responsible when things go astray, which means that the weight falls on the shoulder of the individuals.


The danger behind the lengthy cycle of reviews and improvements needed to contribute is death by a thousands cuts. The contributor looses motivation, and no longer finds the energy to finish the work.

How about users?

This article is focused on developers. Yet, users are also an important part of the discussion around open source.

Often communication failures with users are due to frustration. Frustration of being unable to use the software, of hitting a bug, of seeing an important issue still not addressed. This frustration stems from incorrect expectations, which can often be traced to misunderstanding of the processes and the dynamics. Managing expectations is important to improve the dialogue, via the documentation, via notes on the issue tracker.


Communication is hard

Communication is hard: messages are sometimes received differently than we would like. Overworked people discussing very technically challenging issues only makes the matter worse. I have seen people not come across well, while I know they are absolutely lovely and caring.

We are human beings; we are limited; we misunderstand things, and we have feelings.


Emotions - My most vivid memory of a communication failure was when I was a sailing instructor. Trainees that were under my responsibility had put themselves at risk, causing me a lot of worry. During the debrief, I was angry. My failure to convey the messages without emotional loading undermined my leadership on the group, putting everybody at risk for the rest of the week.

Inability to understand the others' point of view, or to communicate ours, can bring in emotions. Emotions most often impedes technical communication.


Limited attention - We, in particular maintainers, are bombarded with email, notifications, text and code to read. As a consequence, it is easy to read things too fast, to stop in the middle, to forget.


Language barriers - Most discussions happen in English; but most of us are not native English speakers. We may hide well our difficulties, but nuances are often lost.


Clique effects - Most interactions in open source are done in writing, with low communication bandwidth. It can be much harder to convince a maintainer on the other side of the world than a colleague in the same room. Schools of thoughts naturally emerge when people work a lot together. These create bubbles, where we have the impression that everything we say is obvious and uncontroversial, and yet we fail to convince people outside of our bubble.


Little things that help

Communication can improved by continuously working on it [3]. It may be obvious to some, but it personally took me many years to learn.

[3] Training materials for managers often discuss communication, and give tricks. I am sure that there are better references than my list below. But that's the best I can do.

Hear the other: exchange

Related presentation

How can we have healthier technical discussions? by Nathaniel J. Smith

Foster multiway discussions - The goal of a technical discussion is to come up to the best solution. Better solutions emerge via confronting different points of view: a single brilliant individual probably cannot find or recognize the best solution alone.

  • Integrate input from as many perspectives as possible.
  • Make sure everyone feels heard.

Don't seek victory - Most important to keep in mind is that giving up on an argument and accepting the other point of view is a perfectly valid option. I naturally biased to think that my view on topics dear to me is the right one. However, I've learned that adopting the view of the other could bring a lot to the social dynamics of a project: we are often debating over details and the bigger benefit comes from moving forward.

In addition, if several very bright people have different conclusions than me about something that they've thought a lot, who am I to disagree?


Convey ideas well: pedagogy

Explain - Give the premises of your thoughts. Unroll your thought processes. People are not sitting in your head, and need to hear not only your conclusion, but how you got there.

Repeat things - Account for the fact that people can forget, and never hesitate to gently restate important points. Reformulating differently can also help explaining.

Keep it short - A typical reading speed is around 200 words a minute. People have limited time and attention span. The greatest help you can provide to your reader is to condense your ideas: let us avoid long threads that require several dozens of minutes to read and digest. There is a tension between this point and the above. My suggestion: remove every word that is not useful, move details to footnotes or postscriptums.


Cater for emotions: tone

Related good advice

Mozilla participation guide, expected behavior section

Stay technical - Always try to get to the technical aspect of the matter, and never the human. Give specific code and wording suggestions. When explaining a decision, give technical arguments, even if they feel obvious to you.

Be positive - Being positive in general helps people feeling happy and motivated. It is well known that positive feedback leads to quicker progress than negative, as revealed eg by studies of class rooms. I am particularly guilty of this: I always forget to say something nice, although I may be super impressed by a contribution. Likewise, avoid negative words when giving feedback (stay technical).

Avoid "you" - The mere use of the pronoun "you" puts the person we are talking to in the center of message. But the message should not be about the person, it should be about the work. It's very easy to react emotionally when it's about us. The passive voice can be useful to avoid putting people as the topic. If the topic is indeed people, sometimes "we" is an adequate substitute for "you".

Assume good faith - There are so many misunderstandings that can happen. People forget things, people make mistakes, people fail to convey their messages. Most often, all these failures are in good faith, and misunderstandings are legitimate. In the rare cases there might possibly be some bad faith, accounting for it will only make communication worse, not better. Along the same line, we should ignore when we feel assaulted or insulted, and avoid replying in kind.

Choose words wisely - The choice of words matter, because they convey implicit messages. In particular, avoid terms that carry judgement values: "good" or "bad". For example "This is done wrong" (note that this sentence already avoids "you"), could be replaced by "There might be more numerically stable / efficient way of doing it" (note also the use of precise technical wording rather than the generic term "better").

Use moderating words - Try to leave room for the other in the discussion. Statements too assertive close the door to different points of view: "this must be changed" (note the lack of "you") should be avoided while "this should be changed" is better. For this reason, this article is riddled with words such as "tend", "often", "feel", "may", "might".

Don't blame someone else - If you feel that there is some pattern that you would like to change, do not point fingers, do not blame others. Rather, point yourself at the center of the story, find an example of this pattern with you, and the message should be that "it is a pattern that we should avoid. "We" is such a powerful term. It unites; it builds a team.

Give your understanding - If you feel that there is a misunderstanding, explain how you are feeling. But do it using "I", and not "you", and acknowledge the subjectivity: "I feel ignored" rather than "you are ignoring me". Even better: only talk about the feeling: "I am loosing motivation, because this is not moving forward", or "I think that am failing to convey why this numerical problem is such an important issue" (note the use of "I think", which avoids casting the situation as necessarily true).

Implicit messages

The four sides view of communication highlights the multiple messages present even in simple statements.


I hope this can be useful. I personally try to apply these rules, because I want to work better with others.

Thanks

to many who gave me feedback: Adrin Jalali, Andreas Mueller, Elizabeth DuPre, Emmanuelle Gouillart, Guillaume Lemaitre, Joel Nothman, Joris Van den Bossche, Nicolas Hug.



PS: note how many times I've used "you" above. I can clearly get better at communication!

27 May 2020 10:00pm GMT

Real Python: How to Write an Installable Django App

In the Django framework, a project refers to the collection of configuration files and code for a particular website. Django groups business logic into what it calls apps, which are the modules of the Django framework. There's plenty of documentation on how to structure your projects and the apps within them, but when it comes time to package an installable Django app, information is harder to find.

In this tutorial, you'll learn how to take an app out of a Django project and package it so that it's installable. Once you've packaged your app, you can share it on PyPI so that others can fetch it through pip install.

In this tutorial, you'll learn:

  • What the differences are between writing stand-alone apps and writing apps inside of projects
  • How to create a setup.cfg file for publishing your Django app
  • How to bootstrap Django outside of a Django project so you can test your app
  • How to test across multiple versions of Python and Django using tox
  • How to publish your installable Django app to PyPI using Twine

Be sure to follow along with the examples by downloading the source code at the link below:

Download the sample code: Click here to get the code you'll use to learn how to write an installable Django app in this tutorial.

Prerequisites

This tutorial requires some familiarity with Django, pip, PyPI, pyenv (or an equivalent virtual environment tool), and tox. To learn more about these topics, check out:

Starting a Sample Django App in a Project

This tutorial includes a working package to help guide you through the process of making an installable Django app. You can download the source code at the link below:

Download the sample code: Click here to get the code you'll use to learn how to write an installable Django app in this tutorial.

Even if you originally intend to make your Django app available as a package, you're likely to start inside a project. To demonstrate the process of moving from Django project to installable Django app, I've made two branches available in the repo. The project branch is the starting state of an app inside of a Django project. The master branch is the finished installable app.

You can also download the finished app at the PyPI realpython-django-receipts package page. You can install the package by running pip install realpython-django-receipts.

The sample app is a short representation of the line items on a receipt. In the project branch, you'll find a directory named sample_project that contains a working Django project. The directory looks like this:

sample_project/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   └── views.py
│
├── sample_project/
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
│
├── db.sqlite3
├── manage.py
├── resetdb.sh
└── runserver.sh

The most current version of Django at the time this tutorial was written was 3.0.4, and all testing was done with Python 3.7. None of the steps outlined in this tutorial should be incompatible with earlier versions of Django-I've used these techniques since Django 1.8. However, some changes are necessary if you're using Python 2. To keep the examples simple, I've assumed Python 3.7 across the code base.

Creating the Django Project From Scratch

The sample project and receipts app were created using the Django admin command and some small edits. To start, run the following code inside of a clean virtual environment:

$ python -m pip install Django
$ django-admin startproject sample_project
$ cd sample_project
$ ./manage.py startapp receipts

This creates the sample_project project directory structure and a receipts app subdirectory with template files that you'll use to create your installable Django app.

Next, the sample_project/settings.py file needs a few modifications:

  • Add '127.0.0.1' to the ALLOWED_HOSTS setting so you can test locally.
  • Add 'receipts' to the INSTALLED_APPS list.

You'll also need to register the receipts app's URLs in the sample_project/urls.py file. To do so, add path('receipts/', include('receipts.urls')) to the url_patterns list.

Exploring the Receipts Sample App

Read the full article at https://realpython.com/installable-django-app/ »


[ Improve Your Python With 🐍 Python Tricks 💌 - Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

27 May 2020 2:00pm GMT

Real Python: How to Write an Installable Django App

In the Django framework, a project refers to the collection of configuration files and code for a particular website. Django groups business logic into what it calls apps, which are the modules of the Django framework. There's plenty of documentation on how to structure your projects and the apps within them, but when it comes time to package an installable Django app, information is harder to find.

In this tutorial, you'll learn how to take an app out of a Django project and package it so that it's installable. Once you've packaged your app, you can share it on PyPI so that others can fetch it through pip install.

In this tutorial, you'll learn:

  • What the differences are between writing stand-alone apps and writing apps inside of projects
  • How to create a setup.cfg file for publishing your Django app
  • How to bootstrap Django outside of a Django project so you can test your app
  • How to test across multiple versions of Python and Django using tox
  • How to publish your installable Django app to PyPI using Twine

Be sure to follow along with the examples by downloading the source code at the link below:

Download the sample code: Click here to get the code you'll use to learn how to write an installable Django app in this tutorial.

Prerequisites

This tutorial requires some familiarity with Django, pip, PyPI, pyenv (or an equivalent virtual environment tool), and tox. To learn more about these topics, check out:

Starting a Sample Django App in a Project

This tutorial includes a working package to help guide you through the process of making an installable Django app. You can download the source code at the link below:

Download the sample code: Click here to get the code you'll use to learn how to write an installable Django app in this tutorial.

Even if you originally intend to make your Django app available as a package, you're likely to start inside a project. To demonstrate the process of moving from Django project to installable Django app, I've made two branches available in the repo. The project branch is the starting state of an app inside of a Django project. The master branch is the finished installable app.

You can also download the finished app at the PyPI realpython-django-receipts package page. You can install the package by running pip install realpython-django-receipts.

The sample app is a short representation of the line items on a receipt. In the project branch, you'll find a directory named sample_project that contains a working Django project. The directory looks like this:

sample_project/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   └── views.py
│
├── sample_project/
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
│
├── db.sqlite3
├── manage.py
├── resetdb.sh
└── runserver.sh

The most current version of Django at the time this tutorial was written was 3.0.4, and all testing was done with Python 3.7. None of the steps outlined in this tutorial should be incompatible with earlier versions of Django-I've used these techniques since Django 1.8. However, some changes are necessary if you're using Python 2. To keep the examples simple, I've assumed Python 3.7 across the code base.

Creating the Django Project From Scratch

The sample project and receipts app were created using the Django admin command and some small edits. To start, run the following code inside of a clean virtual environment:

$ python -m pip install Django
$ django-admin startproject sample_project
$ cd sample_project
$ ./manage.py startapp receipts

This creates the sample_project project directory structure and a receipts app subdirectory with template files that you'll use to create your installable Django app.

Next, the sample_project/settings.py file needs a few modifications:

  • Add '127.0.0.1' to the ALLOWED_HOSTS setting so you can test locally.
  • Add 'receipts' to the INSTALLED_APPS list.

You'll also need to register the receipts app's URLs in the sample_project/urls.py file. To do so, add path('receipts/', include('receipts.urls')) to the url_patterns list.

Exploring the Receipts Sample App

Read the full article at https://realpython.com/installable-django-app/ »


[ Improve Your Python With 🐍 Python Tricks 💌 - Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

27 May 2020 2:00pm GMT

Codementor: Pros and Cons of Python: A Definitive Python Web Development Guide

Python web development is a popular programming language for most companies today. Read here the 5 Important pros and cons of using python for web development.

27 May 2020 11:35am GMT

Codementor: Pros and Cons of Python: A Definitive Python Web Development Guide

Python web development is a popular programming language for most companies today. Read here the 5 Important pros and cons of using python for web development.

27 May 2020 11:35am GMT

IslandT: Return people from a list and dictionary with Python

In this article, we are going to return a list of names that show whether that person is nice or naughty based on True (Nice) or False (Naughty) value from the 'was_nice' key within a list of dictionaries pass into either the get_nice_names function which will only return the name of a person who is nice or get_naughty_names function which will do vise versa.

A list of dictionaries will get passed into below functions that will return either a list of nice person names or a list of naughty person names.

def get_nice_names(people):

    nice_name_list = []
    for nice_list in people:
        if nice_list["was_nice"] == True:
            nice_name_list.append(nice_list["name"])
    return nice_name_list

def get_naughty_names(people):

    bad_name_list = []
    for bad_list in people:
        if bad_list["was_nice"] == False:
            bad_name_list.append(bad_list["name"])
    return bad_name_list

So if you pass in a list of dictionaries into the nice person function you will receive a list of nice person names (was_nice == True) in return. For example,

nice = [{'name': 'Santa', 'was_nice': True}, {'name': 'Warrior reading this kata', 'was_nice': True}] # ['Santa', 'Warrior reading this kata']

Leave your comment below this post if you have another solution to the above problem.

27 May 2020 6:10am GMT

IslandT: Return people from a list and dictionary with Python

In this article, we are going to return a list of names that show whether that person is nice or naughty based on True (Nice) or False (Naughty) value from the 'was_nice' key within a list of dictionaries pass into either the get_nice_names function which will only return the name of a person who is nice or get_naughty_names function which will do vise versa.

A list of dictionaries will get passed into below functions that will return either a list of nice person names or a list of naughty person names.

def get_nice_names(people):

    nice_name_list = []
    for nice_list in people:
        if nice_list["was_nice"] == True:
            nice_name_list.append(nice_list["name"])
    return nice_name_list

def get_naughty_names(people):

    bad_name_list = []
    for bad_list in people:
        if bad_list["was_nice"] == False:
            bad_name_list.append(bad_list["name"])
    return bad_name_list

So if you pass in a list of dictionaries into the nice person function you will receive a list of nice person names (was_nice == True) in return. For example,

nice = [{'name': 'Santa', 'was_nice': True}, {'name': 'Warrior reading this kata', 'was_nice': True}] # ['Santa', 'Warrior reading this kata']

Leave your comment below this post if you have another solution to the above problem.

27 May 2020 6:10am GMT

Mike Driscoll: Python 101 – Learning About Loops

There are many times when you are writing code that you will need to find a way to iterate over something. Perhaps you'll need to iterate over the letters in a string or the objects in a list. The process of iterating over something is done via a loop.

A loop is a programming construct that allows you to iterate over chunks. Those chunks could be the letters in the string or the lines of a file.

In Python, there are two types of loop constructs:

Besides iterating over sequences, you can use a loop to do the same thing multiple times. One example is a web server that is basically an infinite loop. A server waits, listening for a client to send it a message. When it receives the message, the loop will call a function in response.

Another example is the game loop. When you beat a game or lose a game, the game doesn't usually exit. Instead, it will ask you if you want to play again. This is also done by wrapping the entire program in a loop.

In this chapter you will learn how to:

Let's get started by looking at the for loop!

Creating a for Loop

The for loop is the most popular looping construct in Python. A for loop is created using the following syntax:

for x in iterable:
    # do something

Now the code above does nothing. So let's write a for loop that iterates over a list, one item at a time:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     print(item)
... 
1
2
3

In this code, you create a list with three integers in it. Next you create a for loop that says "for each item in my list, print out the item".

Of course, most of the time you will actually want to do something to the item. For example, you might want to double it:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     print(f'{item * 2}')
... 
2
4
6

Or you might want to only print out only the even-numbered items:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     if item % 2 == 0:
...         print(f'{item} is even')
... 
2 is even

Here you use the modulus operator, %, to find the remainder of the item divided by 2. If the remainder is 0, then you know that an item is an even number.

You can use loops and conditionals and any other Python construct to create complex pieces of code that are only limited by your imagination.

Let's learn what else you can loop over besides lists.

Looping Over a String

One of the differences of the for loop in Python versus other programming languages is that you can iterate over any sequence. So you can iterate over other data types.

Let's look at iterating over a string:

>>> my_str = 'abcdefg'
>>> for letter in my_str:
...     print(letter)
... 
a
b
c
d
e
f
g

This shows you how easy it is to iterate over a string.

Now let's try iterating over another common data type!

Looping Over a Dictionary

Python dictionaries also allow looping. By default, when you loop over a dictionary, you will loop over its keys:

>>> users = {'mdriscoll': 'password', 'guido': 'python', 'steve': 'guac'}
>>> for user in users:
...     print(user)
... 
mdriscoll
guido
steve

You can loop over both the key and the value of a dictionary if you make use of its items() method:

>>> users = {'mdriscoll': 'password', 'guido': 'python', 'steve': 'guac'}
>>> for user, password in users.items():
...     print(f"{user}'s password is {password}")
... 
mdriscoll's password is password
guido's password is python
steve's password is guac

In this example, you specify that you want to extract the user and the password in each iteration. As you might recall, the items() method returns a view that is formatted like a list of tuples. Because of that, you can extract each key: value pair from this view and print them out.

This leads us to looping over tuples and getting out individual items from a tuple while looping!

Extracting Multiple Values in a Tuple While Looping

Sometimes you will need to loop over a list of tuples and get each item within the tuple. It sounds kind of weird, but you will find that it is a fairly common programming task.

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for number, fruit in list_of_tuples:
...     print(f'{number} - {fruit}')
... 
1 - banana
2 - apple
3 - pear

To get this to work, you take advantage of the fact that you know each tuple has two items in it. Since you know the format of the list of tuples ahead of time, you know how to extract the values.

If you hadn't extracted the items individually from the tuples, you would have ended up with this kind of output:

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for item in list_of_tuples:
...     print(item)
... 
(1, 'banana')
(2, 'apple')
(3, 'pear')

This is probably not what you expected. You will usually want to extract an item from the tuple or perhaps multiple items, rather than extracting the entire tuple.

Now let's discover another useful way to loop!

Using enumerate with Loops

Python comes with a built-in function called enumerate. This function takes in an iterator or sequence, like a string or list, and returns a tuple in the form of (position, item).

This allows you to know the position of the item in the sequence easily while looping over the sequence.

Here's an example:

>>> my_str = 'abcdefg'
>>> for pos, letter in enumerate(my_str):
...     print(f'{pos} - {letter}')
... 
0 - a
1 - b
2 - c
3 - d
4 - e
5 - f
6 - g

Now let's look at the other type of loop that Python supports!

Creating a while Loop

Python has one other type of looping construct that is called the while loop. A while loop is created with the keyword while followed by an expression. In other words, while loops will run until a specific condition is met.

Let's take a look at how these loops work:

>>> count = 0
>>> while count < 10:
...     print(count)
...     count += 1

This loop is formulated in much the same way as a conditional statement. You tell Python that you want the loop to run as long as the count is less than 10. Inside of the loop, you print out the current count and then you increment the count by one.

If you forgot to increment the count, the loop would run until you stop or terminate the Python process.

You can create an infinite loop by making that mistake or you could do something like this:

Since the expression is always True, this code will print out the string, "Program running", forever or until you kill the process.

Breaking Out of a Loop

Sometimes you want to stop a loop early. For example, you might want to loop until you find something specific. A good use case would be looping over the lines in a text file and stopping when you find the first occurrence of a particular string.

To stop a loop early, you can use the keyword break:

>>> count = 0
>>> while count < 10:
...     if count == 4:
...         print(f'{count=}')
...         break
...     print(count)
...     count += 1
... 
0
1
2
3
count=4

In this example, you want the loop to stop when the count reaches 4. To make that happen, you add a conditional statement that checks if count equals 4. When it does, you print out that the count equals 4 and then use the break statement to break out of the loop.

You can also use break in a for loop:

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for number, fruit in list_of_tuples:
...     if fruit == 'apple':
...         print('Apple found!')
...         break
...     print(f'{number} - {fruit}')
... 
1 - banana
Apple found!

For this example, you want to break out of the loop when you find an apple. Otherwise you print out what fruit you have found. Since the apple is in the second tuple, you will never get to the third one.

When you use break, the loop will only break out of the innermost loop that the break statement is in.

You can use break to help control the flow of the program. In fact, conditional statements along with break are known as flow control statements.

Another statement you can use to control the flow of your code is continue. Let's look at that next!

Using continue

The continue statement is used for continuing to the next iteration in the loop. You can use continue to skip over something.

Let's write a loop that skips over even numbers:

>>> for number in range(2, 12):
...     if number % 2 == 0:
...         continue
...     print(number)
... 
3
5
7
9
11

In this code, you loop over a range of numbers starting at 2 and ending at 11. For each number in this range, you use the modulus operator, %, to get the remainder of the number divided by 2. If the remainder is zero, it's an even number and you use the continue statement to continue to the next value in the sequence. This effectively skips even numbers so that you only print out the odd ones.

You can use clever conditional statements to skip over any number of things in a sequence by using the continue statement.

Loops and the else Statement

A little known fact about Python loops is that you can add an else statement to them like you do with an if/else statement. The else statement only gets executed when no break statement occurs.

Another way to look at it is that the else statement only executes if the loop completes successfully.

The primary use case for the else statement in a loop is for searching for an item in a sequence. You would use the else statement to raise an exception if the item was not found.

Let's look at a quick example:

>>> my_list = [1, 2, 3]
>>> for number in my_list:
...     if number == 4:
...         print('Found number 4!')
...         break
...     print(number)
... else:
...     print('Number 4 not found')
... 
1
2
3
Number 4 not found

This example loops over a list of three integers. It looks for the number 4 and will break out of the loop if it is found. If that number is not found, then the else statement will execute and let you know.

Try adding the number 4 to the list and then re-run the code:

>>> my_list = [1, 2, 3, 4]
>>> for number in my_list:
...     if number == 4:
...         print('Found number 4')
...         break
...     print(number)
... else:
...     print('Number 4 not found')
... 
1
2
3
Found number 4

A more proper way of doing this would be to raise an exception rather than printing a message.

Nesting Loops

Loops can also be nested inside of each other. There are many reasons to nest loops. One of the most common reasons is to unravel a nested data structure.

Let's use a nested list for your example:

>>> nested = [['mike', 12], ['jan', 15], ['alice', 8]]
>>> for lst in nested:
...     print(f'List = {lst}')
...     for item in lst:
...         print(f'Item -> {item}')

The outer loop will extract each nested list and print it out as well. Then in the inner loop, your code will extract each item within the nested list and print it out.

If you run this code, you should see output that looks like this:

List = ['mike', 12]
Item -> mike
Item -> 12
List = ['jan', 15]
Item -> jan
Item -> 15
List = ['alice', 8]
Item -> alice
Item -> 8

This type of code is especially useful when the nested lists are of varying lengths. You may need to do extra processing on the lists that have extra data or not enough data in them, for example.

Wrapping Up

Loops are very helpful for iterating over data. In this article, you learned about Python's two looping constructs:

You also learned about flow control using the break and continue statements. Finally you learned how to use else in your loops and why you might want to nest loops.

With a little practice, you will soon become quite adept at using loops in your own code!

The post Python 101 - Learning About Loops appeared first on The Mouse Vs. The Python.

27 May 2020 5:05am GMT

Mike Driscoll: Python 101 – Learning About Loops

There are many times when you are writing code that you will need to find a way to iterate over something. Perhaps you'll need to iterate over the letters in a string or the objects in a list. The process of iterating over something is done via a loop.

A loop is a programming construct that allows you to iterate over chunks. Those chunks could be the letters in the string or the lines of a file.

In Python, there are two types of loop constructs:

Besides iterating over sequences, you can use a loop to do the same thing multiple times. One example is a web server that is basically an infinite loop. A server waits, listening for a client to send it a message. When it receives the message, the loop will call a function in response.

Another example is the game loop. When you beat a game or lose a game, the game doesn't usually exit. Instead, it will ask you if you want to play again. This is also done by wrapping the entire program in a loop.

In this chapter you will learn how to:

Let's get started by looking at the for loop!

Creating a for Loop

The for loop is the most popular looping construct in Python. A for loop is created using the following syntax:

for x in iterable:
    # do something

Now the code above does nothing. So let's write a for loop that iterates over a list, one item at a time:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     print(item)
... 
1
2
3

In this code, you create a list with three integers in it. Next you create a for loop that says "for each item in my list, print out the item".

Of course, most of the time you will actually want to do something to the item. For example, you might want to double it:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     print(f'{item * 2}')
... 
2
4
6

Or you might want to only print out only the even-numbered items:

>>> my_list = [1, 2, 3]
>>> for item in my_list:
...     if item % 2 == 0:
...         print(f'{item} is even')
... 
2 is even

Here you use the modulus operator, %, to find the remainder of the item divided by 2. If the remainder is 0, then you know that an item is an even number.

You can use loops and conditionals and any other Python construct to create complex pieces of code that are only limited by your imagination.

Let's learn what else you can loop over besides lists.

Looping Over a String

One of the differences of the for loop in Python versus other programming languages is that you can iterate over any sequence. So you can iterate over other data types.

Let's look at iterating over a string:

>>> my_str = 'abcdefg'
>>> for letter in my_str:
...     print(letter)
... 
a
b
c
d
e
f
g

This shows you how easy it is to iterate over a string.

Now let's try iterating over another common data type!

Looping Over a Dictionary

Python dictionaries also allow looping. By default, when you loop over a dictionary, you will loop over its keys:

>>> users = {'mdriscoll': 'password', 'guido': 'python', 'steve': 'guac'}
>>> for user in users:
...     print(user)
... 
mdriscoll
guido
steve

You can loop over both the key and the value of a dictionary if you make use of its items() method:

>>> users = {'mdriscoll': 'password', 'guido': 'python', 'steve': 'guac'}
>>> for user, password in users.items():
...     print(f"{user}'s password is {password}")
... 
mdriscoll's password is password
guido's password is python
steve's password is guac

In this example, you specify that you want to extract the user and the password in each iteration. As you might recall, the items() method returns a view that is formatted like a list of tuples. Because of that, you can extract each key: value pair from this view and print them out.

This leads us to looping over tuples and getting out individual items from a tuple while looping!

Extracting Multiple Values in a Tuple While Looping

Sometimes you will need to loop over a list of tuples and get each item within the tuple. It sounds kind of weird, but you will find that it is a fairly common programming task.

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for number, fruit in list_of_tuples:
...     print(f'{number} - {fruit}')
... 
1 - banana
2 - apple
3 - pear

To get this to work, you take advantage of the fact that you know each tuple has two items in it. Since you know the format of the list of tuples ahead of time, you know how to extract the values.

If you hadn't extracted the items individually from the tuples, you would have ended up with this kind of output:

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for item in list_of_tuples:
...     print(item)
... 
(1, 'banana')
(2, 'apple')
(3, 'pear')

This is probably not what you expected. You will usually want to extract an item from the tuple or perhaps multiple items, rather than extracting the entire tuple.

Now let's discover another useful way to loop!

Using enumerate with Loops

Python comes with a built-in function called enumerate. This function takes in an iterator or sequence, like a string or list, and returns a tuple in the form of (position, item).

This allows you to know the position of the item in the sequence easily while looping over the sequence.

Here's an example:

>>> my_str = 'abcdefg'
>>> for pos, letter in enumerate(my_str):
...     print(f'{pos} - {letter}')
... 
0 - a
1 - b
2 - c
3 - d
4 - e
5 - f
6 - g

Now let's look at the other type of loop that Python supports!

Creating a while Loop

Python has one other type of looping construct that is called the while loop. A while loop is created with the keyword while followed by an expression. In other words, while loops will run until a specific condition is met.

Let's take a look at how these loops work:

>>> count = 0
>>> while count < 10:
...     print(count)
...     count += 1

This loop is formulated in much the same way as a conditional statement. You tell Python that you want the loop to run as long as the count is less than 10. Inside of the loop, you print out the current count and then you increment the count by one.

If you forgot to increment the count, the loop would run until you stop or terminate the Python process.

You can create an infinite loop by making that mistake or you could do something like this:

Since the expression is always True, this code will print out the string, "Program running", forever or until you kill the process.

Breaking Out of a Loop

Sometimes you want to stop a loop early. For example, you might want to loop until you find something specific. A good use case would be looping over the lines in a text file and stopping when you find the first occurrence of a particular string.

To stop a loop early, you can use the keyword break:

>>> count = 0
>>> while count < 10:
...     if count == 4:
...         print(f'{count=}')
...         break
...     print(count)
...     count += 1
... 
0
1
2
3
count=4

In this example, you want the loop to stop when the count reaches 4. To make that happen, you add a conditional statement that checks if count equals 4. When it does, you print out that the count equals 4 and then use the break statement to break out of the loop.

You can also use break in a for loop:

>>> list_of_tuples = [(1, 'banana'), (2, 'apple'), (3, 'pear')]
>>> for number, fruit in list_of_tuples:
...     if fruit == 'apple':
...         print('Apple found!')
...         break
...     print(f'{number} - {fruit}')
... 
1 - banana
Apple found!

For this example, you want to break out of the loop when you find an apple. Otherwise you print out what fruit you have found. Since the apple is in the second tuple, you will never get to the third one.

When you use break, the loop will only break out of the innermost loop that the break statement is in.

You can use break to help control the flow of the program. In fact, conditional statements along with break are known as flow control statements.

Another statement you can use to control the flow of your code is continue. Let's look at that next!

Using continue

The continue statement is used for continuing to the next iteration in the loop. You can use continue to skip over something.

Let's write a loop that skips over even numbers:

>>> for number in range(2, 12):
...     if number % 2 == 0:
...         continue
...     print(number)
... 
3
5
7
9
11

In this code, you loop over a range of numbers starting at 2 and ending at 11. For each number in this range, you use the modulus operator, %, to get the remainder of the number divided by 2. If the remainder is zero, it's an even number and you use the continue statement to continue to the next value in the sequence. This effectively skips even numbers so that you only print out the odd ones.

You can use clever conditional statements to skip over any number of things in a sequence by using the continue statement.

Loops and the else Statement

A little known fact about Python loops is that you can add an else statement to them like you do with an if/else statement. The else statement only gets executed when no break statement occurs.

Another way to look at it is that the else statement only executes if the loop completes successfully.

The primary use case for the else statement in a loop is for searching for an item in a sequence. You would use the else statement to raise an exception if the item was not found.

Let's look at a quick example:

>>> my_list = [1, 2, 3]
>>> for number in my_list:
...     if number == 4:
...         print('Found number 4!')
...         break
...     print(number)
... else:
...     print('Number 4 not found')
... 
1
2
3
Number 4 not found

This example loops over a list of three integers. It looks for the number 4 and will break out of the loop if it is found. If that number is not found, then the else statement will execute and let you know.

Try adding the number 4 to the list and then re-run the code:

>>> my_list = [1, 2, 3, 4]
>>> for number in my_list:
...     if number == 4:
...         print('Found number 4')
...         break
...     print(number)
... else:
...     print('Number 4 not found')
... 
1
2
3
Found number 4

A more proper way of doing this would be to raise an exception rather than printing a message.

Nesting Loops

Loops can also be nested inside of each other. There are many reasons to nest loops. One of the most common reasons is to unravel a nested data structure.

Let's use a nested list for your example:

>>> nested = [['mike', 12], ['jan', 15], ['alice', 8]]
>>> for lst in nested:
...     print(f'List = {lst}')
...     for item in lst:
...         print(f'Item -> {item}')

The outer loop will extract each nested list and print it out as well. Then in the inner loop, your code will extract each item within the nested list and print it out.

If you run this code, you should see output that looks like this:

List = ['mike', 12]
Item -> mike
Item -> 12
List = ['jan', 15]
Item -> jan
Item -> 15
List = ['alice', 8]
Item -> alice
Item -> 8

This type of code is especially useful when the nested lists are of varying lengths. You may need to do extra processing on the lists that have extra data or not enough data in them, for example.

Wrapping Up

Loops are very helpful for iterating over data. In this article, you learned about Python's two looping constructs:

You also learned about flow control using the break and continue statements. Finally you learned how to use else in your loops and why you might want to nest loops.

With a little practice, you will soon become quite adept at using loops in your own code!

The post Python 101 - Learning About Loops appeared first on The Mouse Vs. The Python.

27 May 2020 5:05am GMT

26 May 2020

feedPlanet Python

PyCoder’s Weekly: Issue #422 (May 26, 2020)

#422 - MAY 26, 2020
View in Browser »

The PyCoder’s Weekly Logo


Waiting in asyncio

There are many ways to wait on the results of a coroutine in Python's asyncio framework. They all have different properties and use cases. In this article, they've all been collected in one place to help you figure out which one you need.
HYNEK SCHLAWACK

Latency in Asynchronous Python

Where does latency in asynchronous programs come from? Explore some common mistakes in asyncio programming, the differences between bounded and unbounded queues, and a suggestion for some best practices for programming with asyncio.
CHRTIS WELLONS

Top 10 Python Tools to Make a Developer's Life Easier

alt

Do you ever feel like your list of sprint tasks is infinite? ActiveState has curated the top 10 Python tools to automate the most common, repetitive tasks. From web framework to code template creation, make your life easier and focus on your next big project! Check out the top 10 list →
ACTIVESTATE sponsor

Creating and Modifying PDF Files in Python

Explore the different ways of creating and modifying PDF files in Python. You'll learn how to read and extract text, merge and concatenate files, crop and rotate pages, encrypt and decrypt files, and even create PDFs from scratch.
REAL PYTHON

Preventing SQL Injection: A Django Author's Perspective

How do you prevent a SQL injection? Learn about best practices and tools for checking for vulnerabilities in a codebase in this post from Django creator Jacob Kaplan-Moss.
JACOB KAPLAN-MOSS

Python Job Hunting in a Pandemic

Do you know someone in the Python community who recently was let go from their job due to the pandemic? What does the job landscape currently look like? What are skills and techniques that will help you in your job search? This week Kyle Stratis discusses how he is managing his job search after just being let go from his data engineering job.
REAL PYTHON podcast

Hypermodern Python

Opinionated guide to setting up a productive Python environment, including recommendations for testing, linting, documentation, and continuous integration.
CLAUDIO JOLOWICZ

Discussions

Renaming Months From Number to Name in Pandas

In a DataFrame indexed by month and day, how do you change the month index from numbers, like 1, to a name, like Jan?
STACK OVERFLOW

Why Is -11 % 5 Equal to 4?

How well do you know the modulo operator?
REDDIT

Python Jobs

Senior Python Engineer (Remote)

Gorgias

Senior Python Engineer (Remote - US East Coast)

Enterra Solutions

Kubernetes Engineer: Docker, Python, Linux (Remote)

Travelers Insurance

Senior Python Developer (Washington, DC)

Jobot

More Python Jobs >>>

Articles & Tutorials

Concurrency in Python for Network I/O

What's the best way to request 20,000 URLs in Python? Compare the performance of synchronous requests, multiprocessing, threading, and asyncio to find out which method is the fastest and why.
ABHISHEK NAGEKAR

Keyword-Only Arguments In Python

Keyword-only arguments have been around for a while in Python, but you don't see them used very frequently. Learn why keyword-only arguments are useful, and why you might want to start using them in your own code.
LUKE PLANT

The Cloud Python Developers Love

alt

DigitalOcean is the cloud provider that makes it easy for developers to deploy and scale their applications. From Flask and Django apps to JupyterHub Notebook servers, DigitalOcean enables Python developers to focus their energy on creating software →
DIGITALOCEAN sponsor

Python Community Interview With Russell Keith-Magee

Russell Keith-Magee is a Django core developer and the founder and maintainer of the BeeWare project. In the interview, he talks about his mission to help Python become a feasible option for writing and packaging mobile applications, his PyCon 2020 talk, and more.
REAL PYTHON

Debugging a Containerized Django App in VS Code

Learn how to create a VS Code run configuration to attach to a Docker container and modify manage.py to start a ptvsd (Python Tools for Visual Studio Debug Server) debug server so that you can debug a containerized Django Project in VS Code.
J-O ERIKSSON • Shared by J-O Eriksson

Profile, Understand & Optimize Python Code Performance

You can't improve what you can't measure. Profile and understand Python code's behavior and performance (Wall-time, I/O, CPU, HTTP requests, SQL queries). Browse through appealing graphs. Blackfire.io is now available as Public Beta.
BLACKFIRE sponsor

A Beginner's Guide to Pip

What is pip? In this beginner-friendly course, you'll learn how to use pip, the standard package manager for Python, so that you can install and manage additional packages that are not part of the Python standard library.
REAL PYTHON video

Fil: A New Python Memory Profiler for Data Scientists and Scientists

Fil is a Python memory profiler specifically designed for batch processing applications that read in data, process it, and write out the result.
ITAMAR TURNER-TRAURING

API Request Caching With Python & Redis

REDOWAN DELOWAR • Shared by Redowan Delowar

Projects & Code

einsteinpy: Community Python Package for General Relativity

GITHUB.COM/EINSTEINPY

filprofiler: Memory Profiler for Data Batch Processing Applications Written in Python

GITLAB.COM/PYTHONSPEED

pdm: Modern Python Package Manager With PEP 582 Support

GITHUB.COM/FROSTMING • Shared by Frost Ming

cascadia-code: Fun, New Monospaced Font That Includes Programming Ligatures

GITHUB.COM/MICROSOFT

django-grpc-gramework: gRPC for Django

GITHUB.COM/FENGSP • Shared by Shipeng Feng


Happy Pythoning!
This was PyCoder's Weekly Issue #422.
View in Browser »

alt


[ Subscribe to 🐍 PyCoder's Weekly 💌 - Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

26 May 2020 7:30pm GMT

PyCoder’s Weekly: Issue #422 (May 26, 2020)

#422 - MAY 26, 2020
View in Browser »

The PyCoder’s Weekly Logo


Waiting in asyncio

There are many ways to wait on the results of a coroutine in Python's asyncio framework. They all have different properties and use cases. In this article, they've all been collected in one place to help you figure out which one you need.
HYNEK SCHLAWACK

Latency in Asynchronous Python

Where does latency in asynchronous programs come from? Explore some common mistakes in asyncio programming, the differences between bounded and unbounded queues, and a suggestion for some best practices for programming with asyncio.
CHRTIS WELLONS

Top 10 Python Tools to Make a Developer's Life Easier

alt

Do you ever feel like your list of sprint tasks is infinite? ActiveState has curated the top 10 Python tools to automate the most common, repetitive tasks. From web framework to code template creation, make your life easier and focus on your next big project! Check out the top 10 list →
ACTIVESTATE sponsor

Creating and Modifying PDF Files in Python

Explore the different ways of creating and modifying PDF files in Python. You'll learn how to read and extract text, merge and concatenate files, crop and rotate pages, encrypt and decrypt files, and even create PDFs from scratch.
REAL PYTHON

Preventing SQL Injection: A Django Author's Perspective

How do you prevent a SQL injection? Learn about best practices and tools for checking for vulnerabilities in a codebase in this post from Django creator Jacob Kaplan-Moss.
JACOB KAPLAN-MOSS

Python Job Hunting in a Pandemic

Do you know someone in the Python community who recently was let go from their job due to the pandemic? What does the job landscape currently look like? What are skills and techniques that will help you in your job search? This week Kyle Stratis discusses how he is managing his job search after just being let go from his data engineering job.
REAL PYTHON podcast

Hypermodern Python

Opinionated guide to setting up a productive Python environment, including recommendations for testing, linting, documentation, and continuous integration.
CLAUDIO JOLOWICZ

Discussions

Renaming Months From Number to Name in Pandas

In a DataFrame indexed by month and day, how do you change the month index from numbers, like 1, to a name, like Jan?
STACK OVERFLOW

Why Is -11 % 5 Equal to 4?

How well do you know the modulo operator?
REDDIT

Python Jobs

Senior Python Engineer (Remote)

Gorgias

Senior Python Engineer (Remote - US East Coast)

Enterra Solutions

Kubernetes Engineer: Docker, Python, Linux (Remote)

Travelers Insurance

Senior Python Developer (Washington, DC)

Jobot

More Python Jobs >>>

Articles & Tutorials

Concurrency in Python for Network I/O

What's the best way to request 20,000 URLs in Python? Compare the performance of synchronous requests, multiprocessing, threading, and asyncio to find out which method is the fastest and why.
ABHISHEK NAGEKAR

Keyword-Only Arguments In Python

Keyword-only arguments have been around for a while in Python, but you don't see them used very frequently. Learn why keyword-only arguments are useful, and why you might want to start using them in your own code.
LUKE PLANT

The Cloud Python Developers Love

alt

DigitalOcean is the cloud provider that makes it easy for developers to deploy and scale their applications. From Flask and Django apps to JupyterHub Notebook servers, DigitalOcean enables Python developers to focus their energy on creating software →
DIGITALOCEAN sponsor

Python Community Interview With Russell Keith-Magee

Russell Keith-Magee is a Django core developer and the founder and maintainer of the BeeWare project. In the interview, he talks about his mission to help Python become a feasible option for writing and packaging mobile applications, his PyCon 2020 talk, and more.
REAL PYTHON

Debugging a Containerized Django App in VS Code

Learn how to create a VS Code run configuration to attach to a Docker container and modify manage.py to start a ptvsd (Python Tools for Visual Studio Debug Server) debug server so that you can debug a containerized Django Project in VS Code.
J-O ERIKSSON • Shared by J-O Eriksson

Profile, Understand & Optimize Python Code Performance

You can't improve what you can't measure. Profile and understand Python code's behavior and performance (Wall-time, I/O, CPU, HTTP requests, SQL queries). Browse through appealing graphs. Blackfire.io is now available as Public Beta.
BLACKFIRE sponsor

A Beginner's Guide to Pip

What is pip? In this beginner-friendly course, you'll learn how to use pip, the standard package manager for Python, so that you can install and manage additional packages that are not part of the Python standard library.
REAL PYTHON video

Fil: A New Python Memory Profiler for Data Scientists and Scientists

Fil is a Python memory profiler specifically designed for batch processing applications that read in data, process it, and write out the result.
ITAMAR TURNER-TRAURING

API Request Caching With Python & Redis

REDOWAN DELOWAR • Shared by Redowan Delowar

Projects & Code

einsteinpy: Community Python Package for General Relativity

GITHUB.COM/EINSTEINPY

filprofiler: Memory Profiler for Data Batch Processing Applications Written in Python

GITLAB.COM/PYTHONSPEED

pdm: Modern Python Package Manager With PEP 582 Support

GITHUB.COM/FROSTMING • Shared by Frost Ming

cascadia-code: Fun, New Monospaced Font That Includes Programming Ligatures

GITHUB.COM/MICROSOFT

django-grpc-gramework: gRPC for Django

GITHUB.COM/FENGSP • Shared by Shipeng Feng


Happy Pythoning!
This was PyCoder's Weekly Issue #422.
View in Browser »

alt


[ Subscribe to 🐍 PyCoder's Weekly 💌 - Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

26 May 2020 7:30pm GMT

Real Python: A Beginner's Guide to Pip

What is pip? pip is the standard package manager for Python. It allows you to install and manage additional packages that are not part of the Python standard library. This course is an introduction to pip for new Pythonistas.

In this course, you'll learn about:

As you'll see, the Python community is very active and has created some neat alternatives to pip that you'll learn about later in this course.


[ Improve Your Python With 🐍 Python Tricks 💌 - Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

26 May 2020 2:00pm GMT

Real Python: A Beginner's Guide to Pip

What is pip? pip is the standard package manager for Python. It allows you to install and manage additional packages that are not part of the Python standard library. This course is an introduction to pip for new Pythonistas.

In this course, you'll learn about:

As you'll see, the Python community is very active and has created some neat alternatives to pip that you'll learn about later in this course.


[ Improve Your Python With 🐍 Python Tricks 💌 - Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

26 May 2020 2:00pm GMT

Programiz: Python RegEx

In this tutorial, you will learn about regular expressions (RegEx), and use Python's re module to work with RegEx (with the help of examples).

26 May 2020 10:04am GMT

Programiz: Python RegEx

In this tutorial, you will learn about regular expressions (RegEx), and use Python's re module to work with RegEx (with the help of examples).

26 May 2020 10:04am GMT

Kushal Das: Using Rust to access Internet over Tor via SOCKS proxy 🦀

Tor provides a SOCKS proxy so that you can have any application using the same to connect the Onion network. The default port is 9050. The Tor Browser also provides the same service on port 9150. In this post, we will see how can we use the same SOCKS proxy to access the Internet using Rust.

You can read my previous post to do the same using Python.

Using reqwest and tokio-socks crates

I am using reqwest and tokio-socks crates in this example.

The Cargo.toml file.

[package]
name = "usetor"
version = "0.1.0"
authors = ["Kushal Das <mail@kushaldas.in>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "0.2", features = ["macros"] }
reqwest = { version = "0.10.4", features = ["socks", "json"] }
serde_json = "1.0.53"

The source code:

use reqwest;
use tokio;
use serde_json;

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let proxy = reqwest::Proxy::all("socks5://127.0.0.1:9050").unwrap();
    let client = reqwest::Client::builder()
        .proxy(proxy)
        .build().unwrap();

    let res = client.get("https://httpbin.org/get").send().await?;
    println!("Status: {}", res.status());

    let text: serde_json::Value = res.json().await?;
    println!("{:#?}", text);

    Ok(())
}

Here we are converting the response data into JSON using serde_json. The output looks like this.

✦ ❯ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/usetor`
Status: 200 OK
Object({
    "args": Object({}),
    "headers": Object({
        "Accept": String(
            "*/*",
        ),
        "Host": String(
            "httpbin.org",
        ),
        "X-Amzn-Trace-Id": String(
            "Root=1-5ecc9e03-09dc572e6db5357f28eecf47",
        ),
    }),
    "origin": String(
        "51.255.45.144",
    ),
    "url": String(
        "https://httpbin.org/get",
    ),
})


Instead of any normal domain, you can also connect to any .onion domain via the same proxy.

26 May 2020 9:46am GMT

Kushal Das: Using Rust to access Internet over Tor via SOCKS proxy 🦀

Tor provides a SOCKS proxy so that you can have any application using the same to connect the Onion network. The default port is 9050. The Tor Browser also provides the same service on port 9150. In this post, we will see how can we use the same SOCKS proxy to access the Internet using Rust.

You can read my previous post to do the same using Python.

Using reqwest and tokio-socks crates

I am using reqwest and tokio-socks crates in this example.

The Cargo.toml file.

[package]
name = "usetor"
version = "0.1.0"
authors = ["Kushal Das <mail@kushaldas.in>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "0.2", features = ["macros"] }
reqwest = { version = "0.10.4", features = ["socks", "json"] }
serde_json = "1.0.53"

The source code:

use reqwest;
use tokio;
use serde_json;

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let proxy = reqwest::Proxy::all("socks5://127.0.0.1:9050").unwrap();
    let client = reqwest::Client::builder()
        .proxy(proxy)
        .build().unwrap();

    let res = client.get("https://httpbin.org/get").send().await?;
    println!("Status: {}", res.status());

    let text: serde_json::Value = res.json().await?;
    println!("{:#?}", text);

    Ok(())
}

Here we are converting the response data into JSON using serde_json. The output looks like this.

✦ ❯ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/usetor`
Status: 200 OK
Object({
    "args": Object({}),
    "headers": Object({
        "Accept": String(
            "*/*",
        ),
        "Host": String(
            "httpbin.org",
        ),
        "X-Amzn-Trace-Id": String(
            "Root=1-5ecc9e03-09dc572e6db5357f28eecf47",
        ),
    }),
    "origin": String(
        "51.255.45.144",
    ),
    "url": String(
        "https://httpbin.org/get",
    ),
})


Instead of any normal domain, you can also connect to any .onion domain via the same proxy.

26 May 2020 9:46am GMT

Catalin George Festila: Python Qt5 - PyQt5 and PyGame compatibility with source code.

This tutorial tries to solve from the objectives related to solving and stabilizing compatibility errors between PyQt4 and PyQt5 and creating a common interface between PyQt5 and PyGame.There is always the same problem in programming when the developer for some reason has to change classes, methods and functions and reusing the old code is no longer valid.In this case, common or other errors

26 May 2020 9:11am GMT

Catalin George Festila: Python Qt5 - PyQt5 and PyGame compatibility with source code.

This tutorial tries to solve from the objectives related to solving and stabilizing compatibility errors between PyQt4 and PyQt5 and creating a common interface between PyQt5 and PyGame.There is always the same problem in programming when the developer for some reason has to change classes, methods and functions and reusing the old code is no longer valid.In this case, common or other errors

26 May 2020 9:11am GMT

Tryton News: Security Release for issue9351

@ced wrote:

Synopsis

A vulnerability in sao has been found by Benjamin Kunz Mejri at Vulnerability-Lab. But they publish it without using our responsive disclosure procedure so we had to make this fix in the hurry.

With issue9351 , the web client does not escape the HTML tags from user data. This allow cross-site scripting attack which result in session hijacking, persistent phishing attacks, persistent external redirects to malicious source.

Impact

CVSS v3.0 Base Score: 4.6

  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: Required
  • Scope: Unchanged
  • Confidentiality: Low
  • Integrity: Low
  • Availability: None

Workaround

There is no existing workaround.

Resolution

All affected users should upgrade sao to the latest version.
Affected versions per series:

  • 5.6: <= 5.6.0
  • 5.4: <= 5.4.6
  • 5.2: <= 5.2.14
  • 5.0: <=5.0.22

Non affected versions per series:

  • 5.6: >= 5.6.1
  • 5.4: >= 5.4.7
  • 5.2: >= 5.2.15
  • 5.0: >= 5.0.23

Reference

Concern?

Any security concerns should be reported on the bug-tracker at
https://bugs.tryton.org/ with the type security .

Posts: 1

Participants: 1

Read full topic

26 May 2020 8:30am GMT

Tryton News: Security Release for issue9351

@ced wrote:

Synopsis

A vulnerability in sao has been found by Benjamin Kunz Mejri at Vulnerability-Lab. But they publish it without using our responsive disclosure procedure so we had to make this fix in the hurry.

With issue9351 , the web client does not escape the HTML tags from user data. This allow cross-site scripting attack which result in session hijacking, persistent phishing attacks, persistent external redirects to malicious source.

Impact

CVSS v3.0 Base Score: 4.6

  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: Required
  • Scope: Unchanged
  • Confidentiality: Low
  • Integrity: Low
  • Availability: None

Workaround

There is no existing workaround.

Resolution

All affected users should upgrade sao to the latest version.
Affected versions per series:

  • 5.6: <= 5.6.0
  • 5.4: <= 5.4.6
  • 5.2: <= 5.2.14
  • 5.0: <=5.0.22

Non affected versions per series:

  • 5.6: >= 5.6.1
  • 5.4: >= 5.4.7
  • 5.2: >= 5.2.15
  • 5.0: >= 5.0.23

Reference

Concern?

Any security concerns should be reported on the bug-tracker at
https://bugs.tryton.org/ with the type security .

Posts: 1

Participants: 1

Read full topic

26 May 2020 8:30am GMT

S. Lott: Modern Python Cookbook 2nd ed -- big milestone

Whew.

Chapter rewrites finished.

Technical reviews in process.

Things are going pretty well. Look for Packt to publish this in the next few months. Details will be posted.

Now. For LinkedIn Learning course recordings.

26 May 2020 8:00am GMT

S. Lott: Modern Python Cookbook 2nd ed -- big milestone

Whew.

Chapter rewrites finished.

Technical reviews in process.

Things are going pretty well. Look for Packt to publish this in the next few months. Details will be posted.

Now. For LinkedIn Learning course recordings.

26 May 2020 8:00am GMT

25 May 2020

feedPlanet Python

Podcast.__init__: Dependency Management Improvements In Pip's Resolver

Dependency management in Python has taken a long and winding path, which has led to the current dominance of Pip. One of the remaining shortcomings is the lack of a robust mechanism for resolving the package and version constraints that are necessary to produce a working system. Thankfully, the Python Software Foundation has funded an effort to upgrade the dependency resolution algorithm and user experience of Pip. In this episode the engineers working on these improvements, Pradyun Gedam, Tzu-Ping Chung, and Paul Moore, discuss the history of Pip, the challenges of dependency management in Python, and the benefits that surrounding projects will gain from a more robust resolution algorithm. This is an exciting development for the Python ecosystem, so listen now and then provide feedback on how the new resolver is working for you.

Summary

Dependency management in Python has taken a long and winding path, which has led to the current dominance of Pip. One of the remaining shortcomings is the lack of a robust mechanism for resolving the package and version constraints that are necessary to produce a working system. Thankfully, the Python Software Foundation has funded an effort to upgrade the dependency resolution algorithm and user experience of Pip. In this episode the engineers working on these improvements, Pradyun Gedam, Tzu-Ping Chung, and Paul Moore, discuss the history of Pip, the challenges of dependency management in Python, and the benefits that surrounding projects will gain from a more robust resolution algorithm. This is an exciting development for the Python ecosystem, so listen now and then provide feedback on how the new resolver is working for you.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, node balancers, a 40 Gbit/s public network, fast object storage, and a brand new managed Kubernetes platform, all controlled by a convenient API you've got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they've got dedicated CPU and GPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show!
  • You listen to this show because you love Python and want to keep your skills up to date, and machine learning is finding its way into every aspect of software engineering. Springboard has partnered with us to help you take the next step in your career by offering a scholarship to their Machine Learning Engineering career track program. In this online, project-based course every student is paired with a Machine Learning expert who provides unlimited 1:1 mentorship support throughout the program via video conferences. You'll build up your portfolio of machine learning projects and gain hands-on experience in writing machine learning algorithms, deploying models into production, and managing the lifecycle of a deep learning prototype. Springboard offers a job guarantee, meaning that you don't have to pay for the program until you get a job in the space. Podcast.__init__ is exclusively offering listeners 20 scholarships of $500 to eligible applicants. It only takes 10 minutes and there's no obligation. Go to pythonpodcast.com/springboard and apply today! Make sure to use the code AISPRINGBOARD when you enroll.
  • Your host as usual is Tobias Macey and today I'm interviewing Tzu-ping Chung, Pradyun Gedam, and Paul Moore about their work to improve the dependency resolution capabilities of Pip and its user experience

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing the focus of the work that you are doing?
    • What is the scope of the work, and what is the established criteria for when it is considered complete?
  • What is your history with working on the Pip source code and what interests you most about this project?
  • What are the main sources or manifestations of technical debt that exist in Pip as of today?
    • How does it currently handle dependency resolution?
  • What are some of the workarounds that developers have had to resort to in the absence of a robust dependency resolver in Pip?
  • How is the new dependency resolver implemented?
    • How has your initial design evolved or shifted as you have gotten further along in its implementation?
  • What are the pieces of information that the resolver will rely on for determining which packages and versions to install? (e.g. will it install setuptools > 45.x in a Python 2 virtualenv?)
  • What are the new capabilities in Pip that will be enabled by this upgrade to the dependency resolver?
  • What projects or features in the encompassing ecosystem will be unblocked with the introduction of this upgrade?
  • What are some of the changes that users will need to make to adopt the updated Pip?
  • How do you anticipate the changes in Pip impacting the viability or adoption of Python and its ecosystem within different communities or industries?
  • What are some of the additional changes or improvements that you would like to see in Pip or other core elements of the Python landscape?
  • What are some of the most interesting, unexpected, or challenging lessons that you have learned while working on these updates to Pip?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don't forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you've learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

25 May 2020 11:32pm GMT

Podcast.__init__: Dependency Management Improvements In Pip's Resolver

Dependency management in Python has taken a long and winding path, which has led to the current dominance of Pip. One of the remaining shortcomings is the lack of a robust mechanism for resolving the package and version constraints that are necessary to produce a working system. Thankfully, the Python Software Foundation has funded an effort to upgrade the dependency resolution algorithm and user experience of Pip. In this episode the engineers working on these improvements, Pradyun Gedam, Tzu-Ping Chung, and Paul Moore, discuss the history of Pip, the challenges of dependency management in Python, and the benefits that surrounding projects will gain from a more robust resolution algorithm. This is an exciting development for the Python ecosystem, so listen now and then provide feedback on how the new resolver is working for you.

Summary

Dependency management in Python has taken a long and winding path, which has led to the current dominance of Pip. One of the remaining shortcomings is the lack of a robust mechanism for resolving the package and version constraints that are necessary to produce a working system. Thankfully, the Python Software Foundation has funded an effort to upgrade the dependency resolution algorithm and user experience of Pip. In this episode the engineers working on these improvements, Pradyun Gedam, Tzu-Ping Chung, and Paul Moore, discuss the history of Pip, the challenges of dependency management in Python, and the benefits that surrounding projects will gain from a more robust resolution algorithm. This is an exciting development for the Python ecosystem, so listen now and then provide feedback on how the new resolver is working for you.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, node balancers, a 40 Gbit/s public network, fast object storage, and a brand new managed Kubernetes platform, all controlled by a convenient API you've got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they've got dedicated CPU and GPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show!
  • You listen to this show because you love Python and want to keep your skills up to date, and machine learning is finding its way into every aspect of software engineering. Springboard has partnered with us to help you take the next step in your career by offering a scholarship to their Machine Learning Engineering career track program. In this online, project-based course every student is paired with a Machine Learning expert who provides unlimited 1:1 mentorship support throughout the program via video conferences. You'll build up your portfolio of machine learning projects and gain hands-on experience in writing machine learning algorithms, deploying models into production, and managing the lifecycle of a deep learning prototype. Springboard offers a job guarantee, meaning that you don't have to pay for the program until you get a job in the space. Podcast.__init__ is exclusively offering listeners 20 scholarships of $500 to eligible applicants. It only takes 10 minutes and there's no obligation. Go to pythonpodcast.com/springboard and apply today! Make sure to use the code AISPRINGBOARD when you enroll.
  • Your host as usual is Tobias Macey and today I'm interviewing Tzu-ping Chung, Pradyun Gedam, and Paul Moore about their work to improve the dependency resolution capabilities of Pip and its user experience

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing the focus of the work that you are doing?
    • What is the scope of the work, and what is the established criteria for when it is considered complete?
  • What is your history with working on the Pip source code and what interests you most about this project?
  • What are the main sources or manifestations of technical debt that exist in Pip as of today?
    • How does it currently handle dependency resolution?
  • What are some of the workarounds that developers have had to resort to in the absence of a robust dependency resolver in Pip?
  • How is the new dependency resolver implemented?
    • How has your initial design evolved or shifted as you have gotten further along in its implementation?
  • What are the pieces of information that the resolver will rely on for determining which packages and versions to install? (e.g. will it install setuptools > 45.x in a Python 2 virtualenv?)
  • What are the new capabilities in Pip that will be enabled by this upgrade to the dependency resolver?
  • What projects or features in the encompassing ecosystem will be unblocked with the introduction of this upgrade?
  • What are some of the changes that users will need to make to adopt the updated Pip?
  • How do you anticipate the changes in Pip impacting the viability or adoption of Python and its ecosystem within different communities or industries?
  • What are some of the additional changes or improvements that you would like to see in Pip or other core elements of the Python landscape?
  • What are some of the most interesting, unexpected, or challenging lessons that you have learned while working on these updates to Pip?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don't forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you've learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

25 May 2020 11:32pm GMT

Luke Plant: Keyword-only arguments in Python

"Keyword-only arguments" is a Python feature that was added in version 3.0. But I've seen and used it much less use than I could have. It is described in PEP 3102, which is pretty readable, but I think the feature could benefit from more exposure with examples and rationale.

To understand keyword-only arguments, we first have to clear up a common misunderstanding about Python positional and keyword arguments. Consider the following function:

def print_greeting(name, use_colors=False):
    # ...

We often think of this as having one positional argument and one keyword argument, and expect that calling code will use it like this:

print_greeting("Mary")
print_greeting("Mary", use_colors=True)
print_greeting("Mary", use_colors=False)  # redundant but explicit

In fact, you can also call this function like the following (although it usually surprises me if I see these forms):

print_greeting("Mary", True)
print_greeting(name="Mary")
print_greeting(use_colors=True, name="Mary")

In other words, both arguments can be passed positionally or by name. Technically you should think about positional or keyword arguments as being defined by the function call and not the function definition. (If you know your computer science terminology, the function definition defines parameters, while it is the function call that defines arguments, which has now confused me about whether the PEP should really have been titled "Keyword-only parameters"…)

The actual distinction between name and use_colors is that use_colors has a default argument value, while name does not. This does introduce limitations in how you can define and how you can call them - for example, the following is illegal:

print_greeting(use_colors=True, "Mary")

Having got that cleared up, we can now separate in our minds two different concerns:

With Python's keyword-only arguments we can address these two independently.

To make it impossible to pass use_colors positionally, we can define it like this:

def print_greeting(name, *, use_colors=False):
    # ...

This brings some advantages:

Once we have these keyword-only arguments, we can also ask "do we actually have good default values?" Let's say we realise that the use of color codes is something that is very important and must be passed every time our print_greeting function is used - we don't have a good default. We can write our function like this:

def print_greeting(name, *, use_colors):
    pass

And call it:

print_greeting("Mary", use_colors=True)

If the caller fails to pass use_colors, they see this error message:

TypeError: print_greeting() missing 1 required keyword-only argument: 'use_colors'

I haven't seen keyword-only arguments that much in the wild, which I think is a shame. But now that Python 2 is much less of a concern for me, I'm going to be making much more use of this feature.

For further reading - the twin of this is positional only parameters, a more recent feature that makes it impossible to pass an argument by name.

25 May 2020 10:09pm GMT

Luke Plant: Keyword-only arguments in Python

"Keyword-only arguments" is a Python feature that was added in version 3.0. But I've seen and used it much less use than I could have. It is described in PEP 3102, which is pretty readable, but I think the feature could benefit from more exposure with examples and rationale.

To understand keyword-only arguments, we first have to clear up a common misunderstanding about Python positional and keyword arguments. Consider the following function:

def print_greeting(name, use_colors=False):
    # ...

We often think of this as having one positional argument and one keyword argument, and expect that calling code will use it like this:

print_greeting("Mary")
print_greeting("Mary", use_colors=True)
print_greeting("Mary", use_colors=False)  # redundant but explicit

In fact, you can also call this function like the following (although it usually surprises me if I see these forms):

print_greeting("Mary", True)
print_greeting(name="Mary")
print_greeting(use_colors=True, name="Mary")

In other words, both arguments can be passed positionally or by name. Technically you should think about positional or keyword arguments as being defined by the function call and not the function definition. (If you know your computer science terminology, the function definition defines parameters, while it is the function call that defines arguments, which has now confused me about whether the PEP should really have been titled "Keyword-only parameters"…)

The actual distinction between name and use_colors is that use_colors has a default argument value, while name does not. This does introduce limitations in how you can define and how you can call them - for example, the following is illegal:

print_greeting(use_colors=True, "Mary")

Having got that cleared up, we can now separate in our minds two different concerns:

With Python's keyword-only arguments we can address these two independently.

To make it impossible to pass use_colors positionally, we can define it like this:

def print_greeting(name, *, use_colors=False):
    # ...

This brings some advantages:

Once we have these keyword-only arguments, we can also ask "do we actually have good default values?" Let's say we realise that the use of color codes is something that is very important and must be passed every time our print_greeting function is used - we don't have a good default. We can write our function like this:

def print_greeting(name, *, use_colors):
    pass

And call it:

print_greeting("Mary", use_colors=True)

If the caller fails to pass use_colors, they see this error message:

TypeError: print_greeting() missing 1 required keyword-only argument: 'use_colors'

I haven't seen keyword-only arguments that much in the wild, which I think is a shame. But now that Python 2 is much less of a concern for me, I'm going to be making much more use of this feature.

For further reading - the twin of this is positional only parameters, a more recent feature that makes it impossible to pass an argument by name.

25 May 2020 10:09pm GMT

Will McGugan: Rich gets Richer

Since my last post on Rich there have been a number of improvements. Here are the major highlights:

Indeterminate progress bars

Progress bars now have an indeterminate state which displays a pulsing animation. This a fairly common thing from graphical UIs, here's what it looks like in the terminal:

To see it in action install Rich and run the following:

python -m rich.progress

Or just watch this video.

You can use this new state to show when something is working, but before you know how many 'steps' are involved. For instance, when making a request over the internet, you won't know how many bytes you have to download until you get a response. Rather than show a progress bar at 0% you can show this pulse effect while waiting on the server.

16.7 million colours in Windows Terminal

Rich now detects the new Windows Terminal and enables truecolor support. Previously colours were downgraded to 8-bit, but now you can enjoy the full 16.7 million colours in Windows.

More text styles

I've added support for the following terminal styles: double underline, framed, encircled, and overlined. Of the new 4 styles, I've only see double underline and overline supported, and only on Ubuntu terminal. The lack of support is why they didn't go in version 1.0, but if your terminal supports these new styles, you're in luck.

I don't even know what frame and encircled is supposed to look like. Let me know if you are using a terminal that supports them (send me a screenshot).

© 2020 Will McGugan

The overline and double underline terminal styles on Ubuntu.

Console markup abbreviations

Rich supported abbreviate styles in markup, for instance "[b]Bold[/b]", but only a single abbreviated style could be applied in a tag. If you wanted bold and italic you would have to write "[b][i]Bold and italic[/i][/b]". Now you can write "[b i]Bold and italic[/b i]".

Fixed

A number of minor fixes and one one serious. There was a possibility of a deadlock in progress bars. If you are using progress bars, please upgrade.

Optimizations

I've made a number of optimizations in rendering styles which have made Rich faster across the board. You're unlikely to notice anything unless you are generating many pages of rich text (perhaps with a large markdown file).

Coverage

Coverage has reached 97% which is not bad at all. To be honest though it is the use of type annotations throughout which gives me the most confidence.

25 May 2020 2:03pm GMT

Will McGugan: Rich gets Richer

Since my last post on Rich there have been a number of improvements. Here are the major highlights:

Indeterminate progress bars

Progress bars now have an indeterminate state which displays a pulsing animation. This a fairly common thing from graphical UIs, here's what it looks like in the terminal:

To see it in action install Rich and run the following:

python -m rich.progress

Or just watch this video.

You can use this new state to show when something is working, but before you know how many 'steps' are involved. For instance, when making a request over the internet, you won't know how many bytes you have to download until you get a response. Rather than show a progress bar at 0% you can show this pulse effect while waiting on the server.

16.7 million colours in Windows Terminal

Rich now detects the new Windows Terminal and enables truecolor support. Previously colours were downgraded to 8-bit, but now you can enjoy the full 16.7 million colours in Windows.

More text styles

I've added support for the following terminal styles: double underline, framed, encircled, and overlined. Of the new 4 styles, I've only see double underline and overline supported, and only on Ubuntu terminal. The lack of support is why they didn't go in version 1.0, but if your terminal supports these new styles, you're in luck.

I don't even know what frame and encircled is supposed to look like. Let me know if you are using a terminal that supports them (send me a screenshot).

© 2020 Will McGugan

The overline and double underline terminal styles on Ubuntu.

Console markup abbreviations

Rich supported abbreviate styles in markup, for instance "[b]Bold[/b]", but only a single abbreviated style could be applied in a tag. If you wanted bold and italic you would have to write "[b][i]Bold and italic[/i][/b]". Now you can write "[b i]Bold and italic[/b i]".

Fixed

A number of minor fixes and one one serious. There was a possibility of a deadlock in progress bars. If you are using progress bars, please upgrade.

Optimizations

I've made a number of optimizations in rendering styles which have made Rich faster across the board. You're unlikely to notice anything unless you are generating many pages of rich text (perhaps with a large markdown file).

Coverage

Coverage has reached 97% which is not bad at all. To be honest though it is the use of type annotations throughout which gives me the most confidence.

25 May 2020 2:03pm GMT

10 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: King Willams Town Bahnhof

Gestern musste ich morgens zur Station nach KWT um unsere Rerservierten Bustickets für die Weihnachtsferien in Capetown abzuholen. Der Bahnhof selber ist seit Dezember aus kostengründen ohne Zugverbindung - aber Translux und co - die langdistanzbusse haben dort ihre Büros.


Größere Kartenansicht




© benste CC NC SA

10 Nov 2011 10:57am GMT

09 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein

Niemand ist besorgt um so was - mit dem Auto fährt man einfach durch, und in der City - nahe Gnobie- "ne das ist erst gefährlich wenn die Feuerwehr da ist" - 30min später auf dem Rückweg war die Feuerwehr da.




© benste CC NC SA

09 Nov 2011 8:25pm GMT

08 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Brai Party

Brai = Grillabend o.ä.

Die möchte gern Techniker beim Flicken ihrer SpeakOn / Klinke Stecker Verzweigungen...

Die Damen "Mamas" der Siedlung bei der offiziellen Eröffnungsrede

Auch wenn weniger Leute da waren als erwartet, Laute Musik und viele Leute ...

Und natürlich ein Feuer mit echtem Holz zum Grillen.

© benste CC NC SA

08 Nov 2011 2:30pm GMT

07 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Lumanyano Primary

One of our missions was bringing Katja's Linux Server back to her room. While doing that we saw her new decoration.

Björn, Simphiwe carried the PC to Katja's school


© benste CC NC SA

07 Nov 2011 2:00pm GMT

06 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nelisa Haircut

Today I went with Björn to Needs Camp to Visit Katja's guest family for a special Party. First of all we visited some friends of Nelisa - yeah the one I'm working with in Quigney - Katja's guest fathers sister - who did her a haircut.

African Women usually get their hair done by arranging extensions and not like Europeans just cutting some hair.

In between she looked like this...

And then she was done - looks amazing considering the amount of hair she had last week - doesn't it ?

© benste CC NC SA

06 Nov 2011 7:45pm GMT

05 Nov 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Mein Samstag

Irgendwie viel mir heute auf das ich meine Blogposts mal ein bischen umstrukturieren muss - wenn ich immer nur von neuen Plätzen berichte, dann müsste ich ja eine Rundreise machen. Hier also mal ein paar Sachen aus meinem heutigen Alltag.

Erst einmal vorweg, Samstag zählt zumindest für uns Voluntäre zu den freien Tagen.

Dieses Wochenende sind nur Rommel und ich auf der Farm - Katja und Björn sind ja mittlerweile in ihren Einsatzstellen, und meine Mitbewohner Kyle und Jonathan sind zu Hause in Grahamstown - sowie auch Sipho der in Dimbaza wohnt.
Robin, die Frau von Rommel ist in Woodie Cape - schon seit Donnerstag um da ein paar Sachen zur erledigen.
Naja wie dem auch sei heute morgen haben wir uns erstmal ein gemeinsames Weetbix/Müsli Frühstück gegönnt und haben uns dann auf den Weg nach East London gemacht. 2 Sachen waren auf der Checkliste Vodacom, Ethienne (Imobilienmakler) außerdem auf dem Rückweg die fehlenden Dinge nach NeedsCamp bringen.

Nachdem wir gerade auf der Dirtroad losgefahren sind mussten wir feststellen das wir die Sachen für Needscamp und Ethienne nicht eingepackt hatten aber die Pumpe für die Wasserversorgung im Auto hatten.

Also sind wir in EastLondon ersteinmal nach Farmerama - nein nicht das onlinespiel farmville - sondern einen Laden mit ganz vielen Sachen für eine Farm - in Berea einem nördlichen Stadteil gefahren.

In Farmerama haben wir uns dann beraten lassen für einen Schnellverschluss der uns das leben mit der Pumpe leichter machen soll und außerdem eine leichtere Pumpe zur Reperatur gebracht, damit es nicht immer so ein großer Aufwand ist, wenn mal wieder das Wasser ausgegangen ist.

Fego Caffé ist in der Hemmingways Mall, dort mussten wir und PIN und PUK einer unserer Datensimcards geben lassen, da bei der PIN Abfrage leider ein zahlendreher unterlaufen ist. Naja auf jeden Fall speichern die Shops in Südafrika so sensible Daten wie eine PUK - die im Prinzip zugang zu einem gesperrten Phone verschafft.

Im Cafe hat Rommel dann ein paar online Transaktionen mit dem 3G Modem durchgeführt, welches ja jetzt wieder funktionierte - und übrigens mittlerweile in Ubuntu meinem Linuxsystem perfekt klappt.

Nebenbei bin ich nach 8ta gegangen um dort etwas über deren neue Deals zu erfahren, da wir in einigen von Hilltops Centern Internet anbieten wollen. Das Bild zeigt die Abdeckung UMTS in NeedsCamp Katjas Ort. 8ta ist ein neuer Telefonanbieter von Telkom, nachdem Vodafone sich Telkoms anteile an Vodacom gekauft hat müssen die komplett neu aufbauen.
Wir haben uns dazu entschieden mal eine kostenlose Prepaidkarte zu testen zu organisieren, denn wer weis wie genau die Karte oben ist ... Bevor man einen noch so billigen Deal für 24 Monate signed sollte man wissen obs geht.

Danach gings nach Checkers in Vincent, gesucht wurden zwei Hotplates für WoodyCape - R 129.00 eine - also ca. 12€ für eine zweigeteilte Kochplatte.
Wie man sieht im Hintergrund gibts schon Weihnachtsdeko - Anfang November und das in Südafrika bei sonnig warmen min- 25°C

Mittagessen haben wir uns bei einem Pakistanischen Curry Imbiss gegönnt - sehr empfehlenswert !
Naja und nachdem wir dann vor ner Stunde oder so zurück gekommen sind habe ich noch den Kühlschrank geputzt den ich heute morgen zum defrosten einfach nach draußen gestellt hatte. Jetzt ist der auch mal wieder sauber und ohne 3m dicke Eisschicht...

Morgen ... ja darüber werde ich gesondert berichten ... aber vermutlich erst am Montag, denn dann bin ich nochmal wieder in Quigney(East London) und habe kostenloses Internet.

© benste CC NC SA

05 Nov 2011 4:33pm GMT

31 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Sterkspruit Computer Center

Sterkspruit is one of Hilltops Computer Centres in the far north of Eastern Cape. On the trip to J'burg we've used the opportunity to take a look at the centre.

Pupils in the big classroom


The Trainer


School in Countryside


Adult Class in the Afternoon


"Town"


© benste CC NC SA

31 Oct 2011 4:58pm GMT

Benedict Stein: Technical Issues

What are you doing in an internet cafe if your ADSL and Faxline has been discontinued before months end. Well my idea was sitting outside and eating some ice cream.
At least it's sunny and not as rainy as on the weekend.


© benste CC NC SA

31 Oct 2011 3:11pm GMT

30 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Nellis Restaurant

For those who are traveling through Zastron - there is a very nice Restaurant which is serving delicious food at reasanable prices.
In addition they're selling home made juices jams and honey.




interior


home made specialities - the shop in the shop


the Bar


© benste CC NC SA

30 Oct 2011 4:47pm GMT

29 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: The way back from J'burg

Having the 10 - 12h trip from J'burg back to ELS I was able to take a lot of pcitures including these different roadsides

Plain Street


Orange River in its beginngings (near Lesotho)


Zastron Anglican Church


The Bridge in Between "Free State" and Eastern Cape next to Zastron


my new Background ;)


If you listen to GoogleMaps you'll end up traveling 50km of gravel road - as it was just renewed we didn't have that many problems and saved 1h compared to going the official way with all it's constructions sites




Freeway


getting dark


© benste CC NC SA

29 Oct 2011 4:23pm GMT

28 Oct 2011

feedPython Software Foundation | GSoC'11 Students

Benedict Stein: Wie funktioniert eigentlich eine Baustelle ?

Klar einiges mag anders sein, vieles aber gleich - aber ein in Deutschland täglich übliches Bild einer Straßenbaustelle - wie läuft das eigentlich in Südafrika ?

Ersteinmal vorweg - NEIN keine Ureinwohner die mit den Händen graben - auch wenn hier mehr Manpower genutzt wird - sind sie fleißig mit Technologie am arbeiten.

Eine ganz normale "Bundesstraße"


und wie sie erweitert wird


gaaaanz viele LKWs


denn hier wird eine Seite über einen langen Abschnitt komplett gesperrt, so das eine Ampelschaltung mit hier 45 Minuten Wartezeit entsteht


Aber wenigstens scheinen die ihren Spaß zu haben ;) - Wie auch wir denn gücklicher Weise mussten wir nie länger als 10 min. warten.

© benste CC NC SA

28 Oct 2011 4:20pm GMT