Understanding Iterator Objects

00:00 In the previous lesson, I introduced you to iterables and their companion iterator object classes. In this lesson, I’ll show you exactly what makes an iterator class an iterator class.

00:12 In lesson two, I explained that the iterator protocol is based on two built-in functions iter() and next(). Under the covers, what those two functions do is invoke corresponding class special methods, double underscore iter, and double underscore next. These kinds of double underscore special methods are sometimes known as magic methods or as dunder methods.

00:35 Dunder being short for double underscore and being easier and more fun to say. Dunder iter() is used in two places: in objects that are iterables and for objects that are iterators.

00:48 An iterables dunder iter() is responsible for returning an iterator object like the one you saw with the list iterator in lesson two. This can be a little confusing.

01:00 Both iterables and iterators use dunder iter(). In an iterable it returns an iterator object for the iterable and for the iterator it returns itself.

01:13 It’s done this way so the for loop or any other construct using the iterator protocol doesn’t have to know the difference between an iterable and an iterator.

01:23 It merely calls iter() in either situation. For an iterable, it gets a newly constructed iterator while with an iterator it gets said the same iterator.

01:34 This keeps the underlying code simple. The protocol doesn’t have to do any introspection, it just calls iter() either way. This has an interesting consequence.

01:44 All iterators are iterables as they implement the dunder iter() method. Not all iterables are iterators, though. Iterables don’t have to implement dunder next().

01:55 That’s what the iterator associated with the iterable is for. All this feels a bit like word salad. Let’s go into the REPL and see if an example helps to clear up this subtle distinction. I’m going to create a class called cats, which is a container for different kinds of cat strings.

02:23 This underscore cats private member in the Cats class contains three types of cats, lions and tigers and not bears. Oh my. Let me instantiate the class and I’ll attempt to iterate over it.

02:40 The Cats class doesn’t implement __iter__, so a type error gets raised telling you this isn’t an iterable. And of course, seeing as the for loop does the same thing,

02:55 I get the same error. Let’s implement a cat iterator that iterates over the Cats class.

03:13 Inside __init__ I need a reference to the thing being iterated over. I’ve gone with the generic name underscore data. When iterating over the cat names I need to know where in the list that’s used as the data structure that I currently am.

03:31 So I’m storing an index value for the list, starting with the first item at zero. To be an iterator, this class needs a dunder iter(), and a dunder next() method. The dunder iter() simply has to return itself.

03:51 Again, this is to make iterables and iterators consistent for the for loop and anyone else using the iterator protocol. Next is well dunder next().

04:03 This is where the actual work of the iteration gets done.

04:12 First, I check whether I’ve reached the end of the list of cat names. If I have, I raise a stop iteration exception to indicate that I’m done.

04:28 This line is where I figure out what value to return. I access the current item in the underscore cats list using my current underscore index. Then I increment the index for next time and return the cats value and that’s it.

04:47 You’ve got a cat iterator class. Lemme try it out.

04:56 Remember, cats itself isn’t iterable, so I have to instantiate the iterator by hand and use that in the for loop. But by doing so, I now have code that works with the for loop printing each of our kitties. Manually instantiating an iterator object is a pain in the butt and it would be far more convenient to make the cats class iterable.

05:19 Let’s see how that’s done with a new class.

05:32 This is a similar __init__, but this time with doggy names. This method isn’t strictly necessary, but it is convenient. __len__ gets called when you use the built-in len function on an object. Implementing this will make our code slightly easier to read a little later on. The length of the dog class is the same as private member data inside.

06:00 I could have hard coded three, but instead of doing that I’ve used len so that way if I add a new doggy name inside of __init__, I don’t have to remember to update this method.

06:13 And here’s what makes the doggy class iterable. The __init__ method. Inside of here, I need to return a new instance of a dog iterator object, which I’ll define in a second.

06:28 Has it been a second? As the dog iterator is only useful with the dog class I’ve made it an inner class hiding it away. This isn’t necessary. You could define it elsewhere, but I like this pattern as it keeps all the logic together.

06:51 Like with the cats iterator, the dogs iterator stores the iterable being iterated and tracks an index.

07:04 Also like the cats iterator __iter__ returns itself. Now comes the __next__.

07:18 In this line, I’m checking whether I’m done iterating. Note that in the cats iterator, I called len on self. _data._cats Whereas here because the dog class implements __len__, I can just use len on the class itself.

07:46 And the rest of the __next__ method is used to return the current item in the iteration. Who let the dogs out? Well there’s a dated musical reference to a mediocre song that has a tendency to stick in your head, Google it, and come back for your apology.

08:02 After that, I’ll introduce you to Rick Rowling. Because I implemented __iter__ on dogs it’s iterable, so calling iter() on it returns our dog iterator object.

08:16 This of course means I can now use it directly in the for loop, unlike with the cats class.

08:26 And there you go, an iterable dogs object. So that’s what makes an iterator object. In the next lesson, I’ll show you some examples where this can actually get used.