Combining Iterators Into Pipelines

00:00 Before you meet any function from itertools, it helps to look at two tools you already know, zip() and map(). They show you what the whole module is about.

00:10 Two terms come up again and again in this course. An iterable is any object that can return its elements one at a time. Lists, strings, and ranges are all iterables.

00:22 An iterator is the object that actually does the work. Each time you call next() on it, you get the next element. And the built-in iter() function turns an iterable into an iterator.

00:34 Open a REPL and see this in action. Start with a list of three numbers. Pass it to iter() to get an iterator. Call next() and you get the first number, one.

00:46 Call it again and you get two. The iterator remembers where it is. If you pass the iterator to list(), it collects whatever is left, which is just three.

00:56 Now the iterator is exhausted. Asking for more gives you an empty list. Under the hood, zip() and map() work with iterators like this one. zip() takes iterables, advances each of their iterators in step, and hands you tuples of matching elements.

01:14 map() applies a function to each element. Here, len() gives you the length of each string. Since zip() returns an iterator, you can feed it straight into map().

01:25 This sums the matching elements of two lists, and you get five, seven, and nine. The docs call this an iterator algebra. Each tool takes iterators in and hands an iterator out.

01:37 Here, zip() pulls one value from each list and builds pairs. Then map() takes each pair and passes it to sum. Nothing gets computed until you ask for a value, and only one pair exists at any moment.

01:51 That’s the idea behind every function in itertools: small building blocks that you plug together into data pipelines. In this lesson, you stepped through an iterator by hand and composed zip() and map() into a small pipeline.

02:06 In fact, Python 2 had lazy versions of these two functions inside itertools, so if you’ve used zip() or map() in Python 3, then you’ve already been using itertools ideas.

02:17 Next up, you’ll see why this matters by splitting a hundred million numbers into groups.