Becoming Iterable

00:00 In the previous lesson, I zipped and sliced. In this lesson, I’ll show you how iterables get made and the functions that help you iterate. You’ve seen plenty of examples of iterables and functions that create objects for iteration.

00:13 Now it’s time to drill down into how all this works. To iterate over a collection or sequence, you need to do two things. First, you call the built-in iter() function to get an iterable object from your collection.

00:27 And then you call the built-in next() function on that iterable object to get the next item of the iteration. You keep calling next() until you get a StopIteration exception, which indicates there is nothing else to iterate upon.

00:42 Under the covers, this is what a for loop does for you. Let’s play like a fancy chef and deconstruct a for loop. First, I need some data to iterate over.

00:57 Now, for the first of our two steps, I’m going to call iter() on this list.

01:05 This returns an iterator object. Specifically, a list iterator, because it was a list I used as my argument. To get the first value from the iterator, I pass it to the next() function.

01:19 Then again, giving me the second value, green.

01:26 Once more for blue.

01:29 And finally, I get the StopIteration exception telling me there is nothing else to iterate over in this iterator. At its core, the list iterator is a stateful object that tracks the position in the iteration that you’re currently within.

01:45 Each time you call next(), the iterator object returns the next possible thing, then advances the position. If that position is past the end of the iteration, it raises StopIteration.

01:59 This next example is something I’ve never actually done myself, but it’s there, so I’m going to cover it. Instead of passing in an iterable to the iter() function, you can pass in a callable and a sentinel.

02:12 When you do it this way, each call to next() calls the callable and checks if the return value from it is the sentinel. Consider this for loop.

02:22 Here, the callable I’m giving iter() is the built-in input() function, a built-in that I’ll cover later, that asks the user for input.

02:30 The sentinel is the string "done". The for loop will continue to iterate until the answer given to input() is done, then the loop stops.

02:40 Let’s try this out.

02:45 Now, input() has been called and it’s waiting for my response. I’ll type hello. That’s the inner of the loop. Once more, input()’s waiting for a response.

02:54 I’ll type goodbye. And one more time. When I type done, the print doesn’t trigger because that’s the sentinel, and that means the for loop gets exited.

03:06 Let’s do this again, but this time in deconstructed style.

03:14 I’ve created the iterator object. Calling next() prompts me to say something. I’ll type hello. And it returns. If I call next() again,

03:27 goodbye, one more time.

03:32 And typing done. And that raises the StopIteration. 99.9% of the time, you’re just going to use a for loop, but every once in a while, you need more control than that.

03:44 I’ve used iter() and next() with parsing code, where I’m processing text and sometimes I want to advance by one token, and sometimes I want to advance by more than one.

03:54 You can achieve the same thing with a continue keyword and some flags in your for loop, but deconstructing the iteration can occasionally result in cleaner code. You’ve seen many functions now that return iterable objects, but just what does that mean?

04:10 Well, an object is iterable if it implements .__iter__() or .__getitem__(). When you call the iter() function asking Python to iterate over an object, it is actually calling your object’s .__iter__() method to get back an iterable.

04:26 Remember when I called iter() on a list? What came back was a list iterable object. That’s what was returned by the list’s .__iter__() method. The .__getitem__() method is a little different. It’s there for accessing specific indexes of a container, typically using square brackets.

04:45 But if you write a .__getitem__() without a .__iter__(), Python takes care of you. It automatically creates an iterable object that iterates over the length of your container using .__getitem__() to access the values.

04:59 Let’s go write a couple of iterable objects. I’m going to write an object that counts to three. Obviously, you should just use range() to do that. So, in fact, that’s how I’m going to cheat and build it, which is silly, but at least you’ll see how .__iter__() works.

05:17 Defining my class.

05:22 And there I’ve declared .__iter__(). Inside of it, I need to return an iterator.

05:30 I’m using iter() on range to do that. Remember, range itself is iterable, not an iterator. So the iter() function is what gets the iterator on the iterable object that comes back from range().

05:44 Now for my list() trick.

05:48 And I get back a count of three. The ThreeCount class is iterable, and so I can use it in a for loop or with iter() and next() like before.

06:12 I can accomplish the same thing by using .__getitem__() instead.

06:19 Another class.

06:24 And there’s the signature for .__getitem__(). This method takes an argument, the index value of the item in the sequence to return. So if I want the first one, I’d be passing in zero as an example.

06:40 Again, I’m cheating and using range(), just slicing it with the index argument. Now if I instantiate it, I can access the zeroth item. Or index three.

06:56 Or I can use the list() trick to iterate over the whole thing.

07:01 Python knows how to iterate on this object because the default .__iter__() method is .__getitem__() aware. Using either of these two dunder methods, you can now write classes that behave as containers, being iterable.

07:16 Most of the built-in functions are synchronous, meaning the calling code blocks until the function returns. Python supports coroutines, one of several ways of doing asynchronous code execution.

07:29 You use coroutines through the asyncio library and the async and await keywords. To support this, there are asynchronous versions of iter() and next().

07:40 They’re named aiter() and anext(), respectively. For a deeper dive on what makes an iterable, this video course and tutorial covers loads more.

07:51 Or if you’d like to learn more about using the asynchronous versions of the functions covered in this lesson, this course and tutorial are good places to start. Next up, a little bit of functional style programming with filter() and map().