Exploring Iterator Specifics

00:00 In the previous lesson I showed you how to write generator functions and generator expressions. This lesson’s a bit of a hodgepodge covering miscellaneous tidbits about iterators.

00:10 Let’s take a moment and talk about some of the limitations of using an iterator. Kind of hinted at all three of these, but it’s worthwhile to take a second and call them out explicitly. First, an iterator object can only be used once. This makes sense if you think about it as the iterator stores the current state of iteration. Once you’re through it, the current state remains done.

00:33 For the most part, this isn’t a big deal. If you instantiate the iterator directly inside the for statement, this will never be a problem for you.

00:41 If you do need to instantiate an iterator into a variable, just realize you can’t loop through it twice.

00:48 If you’ve got an interval that you need to process twice, you need two different iterator objects, one for each pass.

00:56 The second constraint is that iterators only go forwards. There may be a __next__, but there is no __previous__. It just isn’t part of the iterator protocol.

01:08 And finally, you can’t slice an iterator. It doesn’t support indexing. You can only proceed in a sequence. Of course, you could write a clever class that implements the iterator protocol and gets around these constraints.

01:22 You could add a method for resetting your index and you could implement the __ methods necessary for slicing. But strictly speaking, they’re not part of the iterator protocol.

01:31 So you’d just be creating a special case. I’m not against special cases. If you need it, great, do it. No reason not to. Just understand that an out of the box iterator has these three limitations.

01:44 You’ve seen me use iter() and next() directly on an iterator object, but for the most part, you don’t need them. A for loop takes care of this on your behalf.

01:53 There is an occasional case though, where calling next() directly can be helpful. Sometimes you want to skip a value, especially before starting a loop.

02:02 You can do this by using next() before your loop. For example, a CSV file often has a header line before the lines with data. You might want to do something with that line or skip it altogether before going into a loop that goes line by line through the data itself.

02:19 This code shows you exactly that, that first call to next() on the second line eats the first item returned by the iterator. In this case, that first line is the header.

02:29 Then the for loop picks up the iterator starting from the second line.

02:34 I’ve occasionally used this when doing complex data parsing. If you store the iterator in a variable, then use the variable in the for loop.

02:41 You can have calls to next() inside the loop, advancing the iterator without going to the top of the loop. This can be handy if you need to conditionally pull out one or more items or if your data has some lines that get grouped together.

02:54 Now, remember, not too long ago when discussing limitations, I may have said it’s best not just argue your iterators in a variable because they can only be used once.

03:03 This example I just mentioned does the exact opposite, which is fine. It’s a valid use case. Just be mindful that you can’t loop on it gain. There is an optional second argument to the next() function, which is a default value to return if the iteration is stopped.

03:20 When given this optional value, no StopIteration exception gets raised. You can use this to process based on a flagged value instead of catching the exception.

03:30 This example REPL session shows you next() with the return value being zero. Once the squares are done, you’ll just keep getting zero back. I’ve never actually used this feature myself.

03:41 I’m hard-pressed to think of an example where I wouldn’t just want to use the try-except block. But hey, it’s there if you want it.

03:50 Let’s talk a little bit about vocabulary for a second in case you come across these terms in your travels. The for loop is considered definite iteration.

04:00 The definite here, meaning the number of iterations is known. I’ll explain the quotes around known in a second.

04:08 By contrast, an indefinite iteration is one where the number of iterations isn’t known. A while loop is considered indefinite iteration.

04:18 The distinction to me feels kind of arbitrary since a for loop just acts on an iterator and you can write an infinite iterator, could a for loop then be considered indefinite?

04:29 This is into the realm of what is the sound of one hand clapping and not something to get worried about. But I figured you might come across the terminology if you’re reading about iteration so there you go now you know and, and by the way, the sound of one hand, clapping is easy.

04:44 Slap your fingers into your palm, see one hand clapping.

04:50 In the next lesson, I’ll go back to the beginning. Now that you understand iterators, let’s dive a little further into interables themselves.