Revisiting the Iterator Protocol

00:00 Let’s revisit the iterator protocol. And I want to convince you that it’s not just the for loop that relies on it, it’s everywhere where there’s iteration.

00:09 Some places might be obvious, some places you may not have thought of it as an iteration process. And to do this, I’m going to create a simple class. I’m going to call it TestIterable.

00:20 And this class won’t do anything interesting, but it’ll serve as a way of testing things out. Let’s assume we’re going to pass some data to it and let me create the data, self.data attribute.

00:31 And I’m going to ensure that it’s a list. So this will work with anything that can be converted to a list for now, let’s leave it at that. Let me create an instance of TestIterable and let’s create it from a list.

00:45 [2, 4, 6, 8]. seven, eight, it doesn’t matter.

00:49 Remember this class doesn’t do anything interesting at the moment, right? But we’ve created a class.

00:53 Can I loop for value in test? Oops, for value in test, can I do this?

01:04 What’s the answer before I run it?

01:07 The answer is no. Why? Because TestIterable object is not iterable. Even though I’ve put Iterable in the name, that’s not enough to make it iterable.

01:15 Python doesn’t care what I’ve named it.

01:18 In order for test to be iterable, the class, as you’ve seen already, needs to have the .__iter__() dunder method and the .__iter__() dunder method needs to return an iterator.

01:30 And that’s why I’ve ensured that self.data is always a list. Because this means I can convert self.data into, or rather create the iterator from self.data.

01:42 And this is valid. So now TestIterable is iterable because it has a valid .__iter__(), which returns the iterator from self.data.

01:53 Let me try to run the code again, and now it works. 2, 4, 6, 8.

01:59 Let’s make sure that TestIterable has become iterable because of the .__iter__(). You might say, well that’s obvious, right? But we’ll need this in a bit.

02:08 So let me print the .__iter__() method.

02:15 Oops, I can’t type was called.

02:19 And therefore now you can see that the .__iter__() method was called. So we know that we have gone through this dunder method. There’s nothing else happening in the background.

02:29 Okay, good. Let’s try something else. You’ve just covered list comprehensions. So let’s say we want to create a comprehension. output = [value for

02:46 value in test]. You won’t be too surprised. This is not a standard for loop, but this comprehension has the same for statement in it.

02:55 So you won’t be surprised to know that when I run this, the .__iter__() method was called, I’m not printing anything out here, but you know what output it’s going to be, especially if I spell it correctly.

03:06 Doesn’t matter. Output. So a list comprehension also uses the iterator protocol. Maybe that’s not surprising. How about if I try to unpack

03:19 test using the star? The star is the unpacking operator in this case. So it’s going to take each item one at a time. Let’s run this so it works.

03:30 It gives us 2, 4, 6, 8, all printed next to each other because this is the equivalent of saying print(2, 4, 6, 8) and print. Print those next to each other by default.

03:43 But notice that the .__iter__() method was called. So when I’m unpacking, as I’ve done here, I’m also using the iterator protocol. I can also convince you of that if I comment out .__iter__(), and run the code again, print arguments after a star must be an iterable, not TestIterable.

04:06 So it doesn’t like it there, but if I put the .__iter__() back in, then of course it’s going to work again.

04:12 So that’s another case where iteration is happening there. In order to unpack an iterable, in fact, this works with Iterable, it has to be an iterable. Therefore the iterator protocol is used.

04:25 You’ve also seen a reference to map(), one of the built-in Python functions. It’s technically not the function, but we use it like a function in Python.

04:35 So let’s use map() in this case. Now map() needs a callable, let me use str in this case. And the second argument needs to be test,

04:48 and what this does is it takes every item from test and passes it to this function or callable. In this case, this means it’s going to convert it to a string.

05:00 And the map object will then have all of these values converted to string. In this case, let’s now print output won’t work. Let’s do it anyway. But if you’ve seen map() before, you probably know what’s going to happen.

05:16 What matters here is that the .__iter__() method was called, so once again, when I use test with a map(), the iterator protocol, the same iterator protocol using the for loop and the list comprehension and the unpacking is used in map() as well.

05:33 The map object is if, if you want to see what’s in it, in this case, we can just cast it into a list. So now we have 2, 4, 6, 8, but they’re all strings.

05:43 What I want to show you here, we can keep going, right? And you’ll find many other places. So a good exercise is to use this simple class we created and try to see whether you can spot other things in Python that may be using the iterator protocol.

05:59 And if you see this statement listed, then you know that Python is using the .__iter__() for this class and therefore it’s using the iterator protocol.

06:13 So iteration happens not just in the for loop or in list comprehensions. It happens in many places in Python. Sometimes you might not realize initially that there’s iteration happening.

06:24 Whenever you need every item from an iterable, the iterator protocol kicks in.