You know that street game where the performer (con artist?) has three opaque cups and a small ball. He places the cups upside down on the table, with the ball under one of the cups. He quickly shuffles the cups around and then asks the player to guess which cup has the ball. You've seen the game on TV, even if you've not seen it in real life.

Following what's happening when you have a chain of iterators in Python can feel like playing that game. But, unlike the street game, there are no scams when you're playing the iterator game. Let's make sure you'll always win.

I'll keep this article short. I wrote many articles about iterables and iterators. If you need to refresh your memory, have a look at The Anatomy of a for Loop and A One-Way Stream of Data • Iterators in Python (Data Structure Categories #6).

Follow The Data in a Chain of Iterators

Let's keep the example simple. Start with this list in a REPL session:

All code blocks are available in text format at the end of this article • #1

A list is iterable. You can create an iterator from any iterable. Let's create an iterator from this list:

#2

The built-in function iter() creates an iterator from an iterable. Iterators don't contain data. They don't create copies of the data. They're lightweight objects that create a stream. They'll fetch data from the original source, which is the list boring_numbers in this case, as and when needed.

Iterators can only fetch an item once. So, they're a one-way stream. Once you use an item, it's gone from the iterator - but not from the original list, which remains unchanged.

Therefore, first_iter is an iterator that relies on data from the list boring_numbers. But let's not fetch any items from the first_iter iterator. Not yet, anyway.

Create a second iterator. This time, you'll use a generator expression. Generators are iterators, so you create a second iterator with this code:

#3

Note that the expression on the right-hand side of the equals sign is enclosed in parentheses - the round ones, to be clear. This is a generator expression, which creates a generator iterator. Read Pay As You Go • Generate Data Using Generators (Data Structure Categories #7) for more on generators.

As we said, generators are iterators.

The second_iter iterator generates data from first_iter, which is itself an iterator. Iterators are also iterable, which is why you can use them directly in a for clause or anywhere else you'd generally use an iterable. The second_iter iterator will yield the values as floats. But you've not yielded any value from this iterator either. Not yet.

Let's go a step further and create a third iterator, which is also a generator in this case. You build this third iterator from the second one, second_iter:

#4

The generator iterator third_iter yields the sum of 0.5 and the value yielded by second_iter.

Incidentally, I used a "standard" iterator and two generator iterators in this example. However, for the journey we're following in this article, it doesn't matter whether we're using a basic iterator or a generator iterator. If you prefer, you can repeat this exercise with iterators you get from iter() directly.

Support The Python Coding Stack

Don't Blink • Follow the Data

You started with a list called boring_numbers. This data structure contains* the data. It's where the data lives. We'll be following the data in this section. So it's important to know where it's stored!

*Note: Lists, like all data structures, don't really contain data in the purest sense of the word. See What's In A List-Yes, But What's Really In A Python List for more on this. But in general, it's fine to talk about a list 'containing' items of data.

You then create three iterators. The first uses data from boring_numbers. The second iterator uses data from the first. And the third iterator uses data from the second.

But you haven't tried to fetch any value from any of the iterators yet.

Let's look at what each iterator is doing at the moment before you fetch any values. The first iterator, first_iter, is pointing at the first item in boring_numbers. It's ready to read this value and yield it.

The second iterator, second_iter, is pointing at the first item in first_iter. But first_iter doesn't have any data. Iterators don't have their own data. But that's OK. Whenever second_iter needs to fetch the value, it will ask first_iter to fetch and yield its "first" value. I put "first" in quotation marks because you'll see later that this may or may not be the first value.

Finally, third_iter is pointing at the first item in second_iter. The same logic applies. When third_iter needs the first item, it will ask second_iter for its "first" item, and second_iter will need to ask first_iter for its "first" item. And first_iter is pointing at the first item in the list boring_numbers.

Are you with me? Let's complicate things a bit…

Note how your code so far includes the following lines:

#5

None of the iterators has yielded any value. For now.

Let's jumble things up and start by fetching the first value from second_iter:

#6

You ask for the next value in second_iter, which is the first one since you haven't yielded any values yet.

As you've seen earlier, second_iter needs the first value from first_iter. So, behind the scenes, Python calls next(first_iter), which yields the first item from boring_numbers.

So, first_iter reads the first value from boring_numbers, which is the integer 1, and it yields it to second_iter, which then yields the transformed version to the REPL as the return value of next(second_iter). That's why the output is the float 1.0. The first iterator, first_iter, now moves to point at the second item in boring_numbers, ready for when it's needed.

Note that boring_numbers doesn't change in this process. The first item in boring_numbers remains there. It doesn't disappear.

So far, so good?

Continue in the same REPL session and try the following:

#7

You ask third_iter to give you its "next" value. You haven't used third_iter anywhere so far. So, you might expect it to yield the "first" value.

And it does.

But its interpretation of what's the "first" item may be different to what you expect.

Let's follow the data. When you call next(third_iter), the third iterator asks second_iter for its next item. The second iterator, second_iter, relies on first_iter, so it asks first_iter for its next item. And first_iter, as you may recall, is currently pointing at the second item in boring_numbers, which is the integer 2.

So:

1. The first iterator first_iter gets the integer 2 from boring_numbers and yields it to second_iter. And first_iter now points at the third item in boring_numbers.

2. Then, second_iter transforms this value into a float and yields 2.0 to third_iter.

3. Finally, third_iter adds 0.5 to this value and yields 2.5, which is what you see displayed in the REPL.

When you called next(second_iter) earlier in the code, you used up the first item in second_iter, which in turn used up the first item in first_iter. Since this first value is gone and since third_iter depends on the data yielded by second_iter and first_iter, the earlier call to next(second_iter) also affected the iterator that's downstream, third_iter.

What will happen if you call next(first_iter) now? Try to follow the data in your head before trying it out or reading on.

.

.

Have you worked it out?

.

.

Let's run the code:

#8

Although it's the first time you explicitly use first_iter in your code, you already used two of its values when your code yielded values from iterators downstream. Therefore, the next item in first_iter is the third item in boring_numbers, the integer 3.

Let's finish with one more expression, still running in the same REPL session:

#9

You call next(third_iter), which asks second_iter for its next item. And second_iter asks first_iter for its next item. At this stage in the process, first_iter is pointing at the fourth item in the original source of data, which is the list boring_numbers. That's why the output is 4.5.

Independent Iterators

Consider the following code, which is similar to the one you wrote above but has one extra line:

#11

The iterators first_iter and another_first_iter both use the same source of data, boring_numbers. However, they are independent iterators. Note that when you use up some of the elements in first_iter, the independent another_first_iter is not affected. The first time you ask for the first item in another_first_iter, you get the integer 1.

Final Words

Iterators don't contain data. They rely on data that's stored elsewhere. But you can have a chain of iterators, each asking the previous one to yield a value. Weird things can happen if you're not careful. But now you know how to follow the data when you have a chain of iterators.

As a rule of thumb, if you create an iterator that depends on another iterator, you should only use the final iterator to avoid these issues. So, in the example above, you should only yield values from third_iter.

Have a play with this example and make your own chains of iterators, too. And once you're comfortable with this, get ready to be confused again with my next article, which will discuss itertools.tee()!

And next time you pass by someone in the street offering to let you play the three-cups-and-ball game, don't feel overconfident because of your iterator knowledge - it won't help you find the ball.

Code in this article uses Python 3.14

The code images used in this article are created using Snappify. [Affiliate link]

Join The Club, the exclusive area for paid subscribers for more Python posts, videos, a members' forum, and more.

Subscribe now

For more Python resources, you can also visit Real Python-you may even stumble on one of my own articles or courses there!

Also, are you interested in technical writing? You'd like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.

And you can find out more about me at stephengruppetta.com

Further reading related to this article's topic:

• The Anatomy of a for Loop

• A One-Way Stream of Data • Iterators in Python (Data Structure Categories #6)

• Pay As You Go • Generate Data Using Generators (Data Structure Categories #7)

• What's In A List-Yes, But What's Really In A Python List

Appendix: Code Blocks

Code Block #1

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Code Block #2

# ...
first_iter = iter(boring_numbers)

Code Block #3

# ...
second_iter = (float(number) for number in first_iter)

Code Block #4

# ...
third_iter = (num + 0.5 for num in second_iter)

Code Block #5

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
first_iter = iter(boring_numbers)
second_iter = (float(number) for number in first_iter)
third_iter = (num + 0.5 for num in second_iter)

Code Block #6

# ...
next(second_iter)
# 1.0

Code Block #7

# ...
next(third_iter)
# 2.5

Code Block #8

# ...
next(first_iter)
# 3

Code Block #9

# ...
next(third_iter)
# 4.5

Code Block #10

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
first_iter = iter(boring_numbers)
another_first_iter = iter(boring_numbers)
second_iter = (float(number) for number in first_iter)
third_iter = (num + 0.5 for num in second_iter)
next(second_iter)
# 1.0
next(third_iter)
# 2.5
next(first_iter)
# 3
next(third_iter)
# 4.5
next(another_first_iter)
# 1

For more Python resources, you can also visit Real Python-you may even stumble on one of my own articles or courses there!

Also, are you interested in technical writing? You'd like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.

And you can find out more about me at stephengruppetta.com

04 Jun 2026 12:50pm GMT

In this quiz, you'll test your understanding of How to Read User Input From the Keyboard in Python.

By working through this quiz, you'll revisit the input() function, type conversion, error handling with try and except, the getpass module for hidden input, and the PyInputPlus library for automatic validation.

[ Improve Your Python With 🐍 Python Tricks 💌 - Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

04 Jun 2026 12:00pm GMT

In May, we shared the high-level goals of the Python Software Foundation's (PSF) strategic plan and asked for your commentary. Today we are publishing the full draft and opening a three-week community feedback window.

We welcome you to review the full PSF Strategic Plan Community Draft 2026 document, also embedded below.

The feedback window closes on June 25, 2026, End Of Day, Anywhere on Earth. The PSF Board will carefully review all input, use it to refine the final version of the strategic plan, and aims to hold a vote to adopt it in a future board meeting.

What's in the full draft

The earlier blog post covered the six organizational goals and four program goals at a high level. The full draft goes deeper: each program goal includes specific strategic objectives, and the organizational goals include tactical ideas the board developed during the planning process. These tactical ideas are starting points for strategic discussion, not commitments.

This is the first post in a short series. Individual board members will share posts that go into specific parts of the plan in more depth. We want the plan to speak for itself, so these posts will draw directly from the document rather than rewriting it.

What we heard at PyCon US

At PyCon US 2026, the PSF Board held its on-site board meeting, with a portion of that time dedicated to strategy. We also discussed the strategic plan at the Members Lunch, a dedicated Open Space session, and in conversations throughout the conference.

The topic of financial sustainability came up repeatedly, and we hear you. The community is waiting for updated financial information, and typically the Members Lunch at PyCon US is where those details are shared. Staffing changes in our accounting functions made that impossible this year. Publishing the full picture is a priority, and we will share an update as soon as we can. The high-level view is that the PSF is stable for now, but we cannot continue on the current path without making meaningful changes. The strategic plan and the PSF's financial outlook are connected, and we understand that context matters. We are committed to being transparent about both.

We also noticed that conversations naturally moved toward implementation ("How will you do this?"). For this feedback round, we are asking you to focus on the direction itself. Are these the right goals? Are the objectives the right ones? Is anything important missing? Implementation will be shaped by PSF staff over time, and there will be opportunities to weigh in on that, too.

How to give feedback

• Email strategy@python.org to share detailed or private feedback. This is the best way to reach us.
• Discuss thread for open conversation.
• PSF Board Office Hours on the PSF Discord on:
• • June 9, 2026, 9 PM UTC
  • June 23, 2026, 1 PM UTC

The feedback window closes on June 25th. After that, the board will review all feedback received and decide what changes to make to the strategy document in response.

Thank you for your time. We're working on this strategic plan because the Python community deserves a PSF that's deliberate about where it's headed. Your input makes that possible, and we're grateful for your help.

Jannis Leidel, PSF Board Chair, on behalf of the PSF Board of Directors

04 Jun 2026 9:38am GMT

Anything new?

A lot of time has passed since I officially announced that I want to step down from maintaining django-mptt. I started contributing around 2009, tagged the 0.3 release in April 2010, and have been the sole active maintainer since somewhere around 2019. The post about django-tree-queries has more background, but that's not today's topic.

Stepping away isn't easy

For me, abandoning a project is a bit like stepping out of a relationship: negative emotions end up being a somewhat necessary driver, because the absence of positive events alone rarely provides enough force on its own. I get a lot of satisfaction from a job well done, and walking away means letting that go.

Even with time set aside for open source in my work day, I still have to choose where that time goes. django-mptt stopped being where it needed to go.

The sense of entitlement

When a project is obviously unmaintained, asking for free labor is walking a tightrope. It takes real care not to rekindle exactly the frustrations that led maintainers away in the first place.

It takes energy not to clap back when someone is being rude or insensitive in the issue tracker. Asking "Anything new?" on a ticket where the next steps were outlined clearly and obviously nothing happened in the meantime is just one variant of this.

Quietly quitting isn't what I want to do - and as a user of django-mptt myself, I can't really do that either. Taking the high road is the professional choice. But it costs something.

I keep coming back to Mona Eltahawy on refusing to be civil. She's speaking about something quite different, and I'm aware I write this as a white man. The situations aren't the same at all. But she articulated something I haven't managed to put into words as well myself and I like the idea of speaking up and taking the fight to those who awaken these feelings instead of taking the high road.

Doing it with AI

No post these days is complete without the obligatory AI mention, but there's some relevancy to it.

I fixed and closed almost all open django-mptt issues in a two-hour Claude session. I've previously written about using LLMs for open source maintenance, and the productivity gain is real whatever the detractors say. And the quality isn't suddenly getting worse. Code wasn't perfect before either. The test suite allows a certain degree of trust in the result and according to my rules for releasing Open Source software we don't have to require more than that.

It doesn't change the underlying dynamic, though. rsync and outrage illustrates the trap neatly: Tridgell got flooded with AI-generated security reports, used AI to handle them, and then got criticized for using AI. The tools that created the workload aren't allowed to address it. The expectation is that the work has to involve sweat and tears and uncountable unpaid hours.

The common goal should be more and better open source software. What we get as Open Source maintainers is shit from both sides: One side took our free work and trained models on it without asking, the other side complains about the supposedly unethical use of AI while acting in unethical ways themselves.

There's something Kantian about how open source contribution gets framed. Kant's argument was that the only truly moral acts are those driven by duty and good will - not by desire, inclination, or any expectation of compensation. By that logic, I'm only acting morally if I keep going despite the burnout and the entitlement. If I stop, I'm not.

It's bleak. The problems with AI are real. The people controlling the large models are assholes. But I have to work in the world as it is while also trying to change it for the better.

03 Jun 2026 5:00pm GMT

djust 1.0 is here - reactive UI for Django in pure Python. No client state, no JavaScript framework, no build step, no API layer. It brings the proven Phoenix LiveView model to Django with a Rust VDOM on the hot path. Try it live (multi-user, no install) at start.djust.org.

02 Jun 2026 6:00pm GMT

The first time I said "code is cheap" out loud in a meeting, a manager waved at the budget - headcount, salaries, the tooling line - and asked which part of that looked cheap. He wasn't wrong about the number - he was wrong about what it was buying.

02 Jun 2026 10:15am GMT

Let's say you're writing a Python library.

In this library, you have some collection of state that represents "options" or "configuration" for a bunch of operations. Such a set of options is a bundle of potentially ever-increasing complexity. Thus, you will want it to have an extremely minimal compatibility surface, with a very carefully chosen public interface, that is either small, or perhaps nothing at all. Such an object conveys state and might have some private behavior, but all you want consumers to be able to do is build it in very constrained, specific ways, and then pass it along as a parameter to your own APIs.

By way of example, imagine that you're wrapping a library that handles shipping physical packages.

There are a zillion ways to do it ship a package. There are different carriers who can ship it for you. There's air freight, and ground freight, and sea freight. There's overnight shipping. There's the option to require a signature. There's package tracking and certified mail. Suffice it to say, lots of stuff.

If you are starting out to implement such a library, you might need an object called something like ShippingOptions that encapsulates some of this. At the core of your library you might have a function like this:

1
2
3
4
5

async def shipPackage(
        how: ShippingOptions,
        where: Address,
    ) -> ShippingStatus:
    ...

If you are starting out implementing such a library, you know that you're going to get the initial implementation of ShippingOptions wrong; or, at the very least, if not "wrong", then "incomplete". You should not want to commit to an expansive public API with a ton of different attributes until you really understand the problem domain pretty well.

Yet, ShippingOptions is absolutely vital to the rest of your library. You'll need to construct it and pass it to various methods like estimateShippingCost and shipPackage. So you're not going to want a ton of complexity and churn as you evolve it to be more complex.

Worse yet, this object has to hold a ton of state. It's got attributes, maybe even quite complex internal attributes that relate to different shipping services.

Right now, today, you need to add something so you can have "no rush", "standard" and "expedited" options. You can't just put off implementing that indefinitely until you can come up with the perfect shape. What to do?

The tool you want here is the opaque data type design pattern. C is lousy with such things (FILE, pthread_*_t, fd_set, etc). A typedef in a header file can easily achieve this.

But in Python, if you expose a dataclass - or any class, really - even if you keep all your fields private, the constructor is still, inherently, public. You can make it raise an exception or something, but your type checker still won't help your users; it'll still look like it's a normal class.

Luckily, Python typing provides a tool for this: typing.NewType.

Let's review our requirements:

1. We need a type that our client code can use in its type annotations; it needs to be public.
2. They need to be able to consruct it somehow, even if they shouldn't be able to see its attributes or its internal constructor arguments.
3. To express high-level things (like "ship fast") that should stay supported as we add more nuanced and complex configurations in the future (like "ship with the fastest possible option provided by the lowest-cost carrier that supports signature verification").

In order to solve these problems respectively, we will use:

1. a public NewType, which gives us our public name...
2. which wraps a private class with entirely private attributes, to give us an actual data structure, while not exposing the constructor,
3. a set of public constructor functions, which returns our NewType.

When we put that all together, it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

from dataclasses import dataclass
from typing import Literal, NewType

@dataclass
class _RealShipOpts:
    _speed: Literal["fast", "normal", "slow"]

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("fast"))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("normal"))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts("slow"))

As a snapshot in time, this is not all that interesting; we could have just exposed _RealShipOpts as a public class and saved ourselves some time. The fact that this exposes a constructor that takes a string is not a big deal for the present moment. For an initial quick and dirty implementation, we can just do checks like if options._speed == "fast" in our shipping and estimation code.

However, the main thing we are doing here is preserving our flexibility to evolve the related APIs into the future, so let's see how we might do that. For example, let's allow the shipping options to contain a concrete and specific carrier and freight method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

from dataclasses import dataclass
from enum import Enum, auto
from typing import NewType

class Carrier(Enum):
    FedEx = auto()
    USPS = auto()
    DHL = auto()
    UPS = auto()

class Conveyance(Enum):
    air = auto()
    truck = auto()
    train = auto()

@dataclass
class _RealShipOpts:
    _carrier: Carrier
    _freight: Conveyance

ShippingOptions = NewType("ShippingOptions", _RealShipOpts)

def shipFast() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.FedEx, Conveyance.air))

def shipNormal() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.UPS, Conveyance.truck))

def shipSlow() -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(Carrier.USPS, Conveyance.train))

def shippingDetailed(
    carrier: Carrier, conveyance: Conveyance
) -> ShippingOptions:
    return ShippingOptions(_RealShipOpts(carrier, conveyance))

As a NewType, our public ShippingOptions type doesn't have a constructor. Since _RealShipOpts is private, and all its attributes are private, we can completely remove the old versions.

Anything within our shipping library can still access the private variables on ShippingOptions; as a NewType, it's the same type as its base at runtime, so it presents minimal¹ overhead.

Clients outside our shipping library can still call all of our public constructors: shipFast, shipNormal, and shipSlow all still work with the same (as far as calling code knows) signature and behavior.

If you need to build and convey some state within your public API, while avoiding breakages associated with compatibility churn, hopefully this technique can help you do that!

Acknowledgments

Thanks for reading, and thank you to my patrons who are supporting my writing on this blog. If you like what you've read here and you'd like to read more of it, or you'd like to support my various open-source endeavors, you can support my work as a sponsor.

22 May 2026 12:33am GMT

Here is a useful trick that is unreasonably effective for simple computer use goals using modern terminal agents. On macOS, there has been a terminal osascript command since the original release of Mac OS X. All you have to do is suggest your agent use it and it can perform any application control action available in any AppleScript dictionary for any Mac app. No MCP set up or tools required at all. Agents are much more adapt at using rod terminal commands, especially ones that haven't changed in 30 years. Having a computer control interface that hasn't changed in 30 years and has extensive examples in the Internet corpus makes modern models understand how to use these tools basically Effortlessly. macOS locks down these permissions pretty heavily nowadays though, so you will have to grant the application control permission to terminal. But once you have done that, the range of possibilities for commanding applications using natural language is quite extensive. Also, for both Safari and chrome on Mac, you are going to want to turn on JavaScript over AppleScript permission. This basically allows claude or another agent to debug your web applications live for you as you are using them.In chrome, go to the view menu, developer submenu, and choose "Allow JavaScript from Apple events". In Safari, it's under the safari menu, settings, developer, "Allow JavaScript from Apple events". Then you can do something like "Hey Claude, would you Please use osascript to navigate the front chrome tab to hacker news". Once you suggest using OSA script in a session it will figure out pretty quickly what it can do with it. Of course you can ask it to do casual things like open your mail app or whatever. Then you can figure out what other things will work like please click around my web app or check the JavaScript Console for errors. Another very important tips for using modern agents is to try to practice using speech to text. I think speaking might be something like five times faster than typing. It takes a lot of time to get used to, especially after a lifetime of programming by typing, but it's a very interesting and a different experience and once you have a lot of practice It starts to to feel effortless.

04 Apr 2026 1:31pm GMT

I have been using macOS voice control for about three years. First it was a way to reduce pain from excessive computer use. It has been a real struggle. Decades of computer use habits with typing and the mouse are hard to overcome! Text selection manipulation commands work quite well on macOS native apps like apps written in swift or safari with an accessibly tagged webpage. However, many webpages and electron apps (Visual Studio Code) have serious problems manipulating the selection, not working at all when using "select foo" where foo is a word in the text box to select, or off by one errors when manipulating the cursor position or extending the selection. I only recently expanded my repertoire with the "start drag" and "drop" commands, previously having used "Click and hold mouse", "move cursor to x", and "release mouse". Well, now I have discovered that using "start drag x" and "drop x" makes a fantastic text selection method! This is really going to improve my speed. In the long run, I believe computer voice control in general is going to end up being faster than WIMP, but for now the awkwardly rigid command phrasing and the amount of times it misses commands or misunderstands commands still really holds it back. I've been learning the macOS Voice Control specific command set for years now and I still reach for the keyboard and mouse way too often.

16 Mar 2026 11:04am GMT