Functions: Iterables and Iterators
Iterables and Iterators. This section is going to look at functions you can use with iterables and iterators. The first one is going to be
len(), which tells you the length of the iterable that you pass it.
Here’s a list filled with some random-ish numbers. Looking at
len(a) shows us there are
9 elements within that list. We can see that that list does indeed have nine elements. Next up,
all() work on the Boolean values of the iterables that you pass to them. As we’ve seen earlier, any entry can be considered to be
False, but to simplify matters, the three lists are going to explicitly use
False. So in the case of
a, all three entries are
In the case of
b, some of the entries are
True but it has a
False present as well.
And the third list will have all
False values. So these are the three possibilities which are present.
If we look at
any(), we can see that
True because some of them are
True. In fact, all of them are
True. If we look at
any(b), that’s also
True because it has some
True values even though they’re not all
any(c) gives us the result
False because none of them are
The partner of this is
all(). If we look at
all(a), because they’re all
True we get
True, as you can see here.
all(b) gives us
False, because not all of them are because we have one
False value there. And as you can imagine, for
all(c) we also get
False because there are no
True values, let alone all of them being
For the next section, we’re going to define a variable
d as a list of numbers. As you can see, it has nine elements in it.
Let’s look at the
reversed() function. If we define
reversed(d), we’d hoped to get that list out backwards, and if we look at it, we don’t get exactly what we’re expecting.
What you may have been expecting is a reverse version of that list, and instead we get a reverse iterator object. Now, most of the time in your code, that will be okay, but some of the time you may want to see that list—for instance, when you’re debugging or printing it out. So if we do
list(e), you can see that list is
d in reverse.
03:04 But that works how you probably expect, and sometimes you will need to do this, but often within your code, you won’t need to do this because Python will deal with the iterator.
03:15 Instead of casting all of this to a list in one go and outputting it, Python will do it in a more efficient manner and only produce the value which is needed when they’re needed.
03:26 The analogy I can think of is imagine having a machine which could produce cars for you. You’d have two modes it could operate in—one where it produces all the cars you need in one go, and one where it does them on demand.
03:40 That would be fine if you only needed one, but if you needed 3 million of them, you’d need to find somewhere to store them all. In this case, it’s storing all those numbers in memory, and it takes time to create them.
Generally, Python’s create-on-demand system is a more efficient way for it to work and has much lower memory demands, but sometimes dealing with this when you’re starting out means you need to cast things into a list to understand what’s happening behind the scenes. With that out of the way, let’s look at
sorted(). Here we’re defining
f as the sorted version of
If we look at
f, it works the way we’d expect—it produces a list and it’s the elements of
d here which have been sorted. And see, we’ve got two
7s, which is why we’ve got two
This list is exactly as we’d expect. And it’s also possible to reverse
sorted() if you need it to be that way around.
Next up, let’s look at
range() is really useful. You may have already encountered it. If we enter
range() here in the REPL, we get this range from
10, which isn’t that helpful because it doesn’t really tell us what’s going on, but if we cast that into a
list, we can see we get the values
Remember, it’s not including this final number, so it’s
9—we get 10 in total. It’s possible to pass it explicitly the start and end values, and also a third value, stride.
Here we can see a
range(0, 10) with a stride of
2. That gives just the even numbers,
8, and again, it doesn’t include the last number.
range() is extremely useful and you’ve probably used it already, but you may not have heard of these last two,
enumerate(), which are easier to understand in the context of a program, so we’re going to switch over to code now.
Iterables and Iterators:
enumerate(). As a demonstration of this, a simple program is going to be created which is going to print out the list of players of an imaginary team.
A typical thing, we have a list here and I’m going to create a
for player in players:
print(player). Fairly piecemeal stuff, and you’ve probably seen something like this before.
If we see that in action, it works exactly as we’d expect, printing each name on its own line. But quite often it’s important to know which number on the list is which index in the list. The way you would probably do this if you weren’t aware of
enumerate() is to create a
count variable which is initialized before the loop starts.
We’ll print the
count as well as the
player out on each line, and then after that, the
count gets incremented with
count += 1, and that’ll work just fine as we can see here.
But Python provides us with
enumerate(), which makes a much cleaner way to do this same functionality. We’re going to delete the
count variable, we’re going to delete the increment of
count, and then we’re going to wrap
players in the
enumerate() function, which outputs two things. It outputs the normal
player, but it will also output the
count—a zero-indexed counter of the number of times a loop has run. So we have much neater code, you don’t have to worry about initializing or incrementing the
count variable—which everyone has forgotten at some point—and running the code gives us exactly the same output as before, but it’s a much neater piece of code to read.
07:24 It’s easy to see exactly what’s happening, and because it’s standard code, you’ll get used to seeing this and understand it intuitively every time.
Iterables and Iterators:
07:42 In this example, you have two lists which need to be paired together—so in this case, a list of three countries,
07:57 and one which is continents which go with them.
You’ll want to create tuples with these paired together. The way you would initially do this would probably involve a
for loop, so in this case, using an index variable combining
range() and the
len() of the
That index will now allow us to access the
08:36 You can check if that works by running the program,
and as you can see, they’ve been paired correctly:
Now that we know we’re accessing those correctly, we can put these together. So first things first, let’s create a list for them to go in. Here’s our empty
and then we can do
merged.append() and put that in brackets inside there so we create a tuple with those two together.
Finally, we can print out the
merged list, and now we can see that running and we have our list with the appropriately paired countries and continents.
If you’re not aware of the
zip() function, this is probably the kind of thing you would do many times. For there’s a much simpler way to do this, and as we can see, we can use
merged_2, it uses the
zip() function with those two lists
continents, and that’s that entire
for loop replaced.
When we run this, they both produce identical output, but this is a much more complicated equivalent to this. The
zip() part is much simpler than these three lines where you create an empty list and a
for loop. It could have been done with a list comprehension but still would have been complicated. But
zip() works faster, is much easier, and is much less error-prone than the manual approach.
It’s important to remember that this is a zip in terms of a zipper on your clothes, not in terms of a compressed file creator.
zip() is incredibly useful whenever you have lists you need to merge together.
Iterables and Iterators:
next(). The last thing we’re going to look at in this section is the creation of an iterator such as we saw earlier on. It’s possible to create an iterator using any iterable. We’re going to use a list in this example.
It has a method
next(), which pops the next value off of that iterator until it’s exhausted, when you’ll get a
StopIteration exception. Let’s make an example now, so here
a = iter() and we’re just going to pass it a list directly of
[1, 2, 3, 4, 5, 6], to keep it nice and simple.
Now we’ve created our iterator
a and now we can just use
next(a) and we get the next value from it, and then
And once it’s exhausted, if we try
next() again, we get a
StopIteration exception. This is the way that any iterable works behind the scenes with Python, so anything which can take an iterator will work exactly the same way with an iterator you’ve created using the
getting below error for ZIP.py <zip object at 0x00633F80>
Related to above comment:
getting below error for ZIP.py <zip object at 0x00633F80>.
Note this is not an error.
print(merged_2) returns the memory address of the iterator
print(list(merged_2)) returns the content of the iterator
You can also call the zip functionality as following:
Exactly what peterabbeel did on <zip object at 0x036B4F08> I also had to switch mine to print(tuple(merged_2) which then gave me the proper output although it makes it immutable. Still learned a ton from this tutorial.
I didn’t even know some of these functions existed such as
all(). Of course everybody should already know about them since their just built-in functions. But very handy.
Learned quite a lot from this video, thanks.
Hey Darren. In the zip function example using the “merged” for-loop, were there two sets of brackets around the “merged.append…” line due to the argument being a tuple (countries[i], continents[i])?
@leroygcooper That’s correct. You need to wrap both elements in a tuple because the list’s
.append() method expects only a single item, and we want the list to contain pairs.
Can you give me an actual use case where function
reversed() can be used? I understand that python stores it as an object and we have to feed it to a list to retrieve the values. Is there a use case where reversed can be used without a list function? may be in a loop function where i need to print the items in a list, in a reverse order?
reversed() function accepts any iterable object, while the
list.reverse() method is only defined for lists. Additionally, sequences like lists, tuples, or strings support the slicing syntax, e.g.,
text[::-1], which lets you achieve a similar effect by making a reversed copy of the original sequence. On the other hand,
reversed() is lazily evaluated and creates a lightweight view of the original data without allocating new memory. This makes changes to the original data visible in your reversed view. You’ll find much more details about the differences between those methods for reversing sequences in the written tutorial: realpython.com/python-reverse-list/
reversed() in a loop or a comprehension expression, like you said. There’s also a protocol you can leverage in your custom classes:
>>> class CustomClass: ... def __iter__(self): ... yield from [1, 2, 3] ... ... def __reversed__(self): ... yield from [3, 2, 1] >>> instance = CustomClass() >>> for value in instance: ... print(value) ... 1 2 3 >>> for value in reversed(instance): ... print(value) ... 3 2 1
So what exactly is the difference between an Iterable and an Iterator? How do we define them?
e = reversed(d)
is the Iterator and
is the Iterable?
@aquibmir See the Python Glossary for the relevant definitions:
In a nutshell, an iterable is a composite object able to yield its elements in a for-loop. There are all kinds of iterable objects in Python, including strings, tuples, lists, dictionaries, and so on. On the other hand, objects like integers aren’t iterable because you can’t iterate over them in a loop.
An iterator is another type of object whose role is to hide the details of visiting the individual elements of an iterable. It’s straightforward to think of iterating over a sequence like a Python list because each element has an associated index. However, some iterables don’t keep their elements in any particular order. For example, sets, trees, graphs, and other data structures are organized differently in memory, yet they all behave the same when you iterate over them.
You can define your own iterable that hooks into the for-loop by implementing a few special methods in a custom class. If your iterable has a known length and is a sequence indexed by a numeric index or a mapping accessed by a hashable key, then you can implement these two special methods in your custom class:
class Iterable: def __getitem__(self, key_or_index): ... def __len__(self): ...
Otherwise, you can use a more generic iterator protocol by implementing the
.__iter__() special method instead:
class Iterable: def __iter__(self): return Iterator(...)
Moreover, if you’d like to support iteration in reversed order, i.e., when someone calls
reversed() on your iterable, then you can optionally add yet another special method:
class Iterable: def __iter__(self): return Iterator(...) def __reversed__(self): return ReversedIterator(...)
In both cases, you return a new instance of an iterator class, which will handle the iteration. Alternatively, when the iteration has a relatively simple logic, it’s customary to use a generator iterator instead of returning a new instance of a class:
class Iterable1: def __iter__(self): yield from self.items class Iterable2: def __iter__(self): for item in self.items: yield item
If you choose to define an iterator class, then it must implement at least the
.__next__() special method, which supplies the next element of the iterable. However, to comply with the iterator protocol, it should also implement the
.__iter__() method that returns itself:
class Iterator: def __next__(self): ... def __iter__(self): return self
This makes the iterator iterable itself, which is mandatory if you want to use it with a for-loop, which always calls
iter() first, and then keeps calling
next() on the resulting iterator object. To test your iterable and the corresponding iterator, you can simulate a for-loop by calling these functions manually:
>>> class Iterable: ... def __init__(self, *elements): ... self.elements = elements ... def __iter__(self): ... return Iterator(self.elements) >>> class Iterator: ... def __init__(self, elements): ... self.elements = elements ... self.index = -1 ... def __next__(self): ... self.index += 1 ... if self.index < len(self.elements): ... return self.elements[self.index] ... raise StopIteration ... def __iter__(self): ... return self >>> iterable = Iterable("apple", "banana", "orange") >>> iterator = iter(iterable) >>> next(iterator) 'apple' >>> next(iterator) 'banana' >>> next(iterator) 'orange' >>> next(iterator) Traceback (most recent call last): File "<input>", line 1, in <module> next(iterator) File "<input>", line 9, in __next__ raise StopIteration StopIteration
To signal that all elements have been exhausted, an iterator must raise the
StopIteration exception. However, it’s optional since your iterator might be infinite, in which case it will never exhaust iterable’s elements.
Note that iterators are typically one-use only, which means that once they reach the last element in the iterable, they’re no longer useful. You must create a brand new iterator instance, which will point the first element in an iterable.
Become a Member to join the conversation.
alanhrosenthal on March 21, 2020
Very complete. Very easy to understand.