Iterators and Iterables in Python: Run Efficient Iterations

What Is an Iterator in Python?

In Python, an iterator is an object that allows you to iterate over collections of data, such as lists, tuples, dictionaries, and sets.

Python iterators implement the iterator design pattern, which allows you to traverse a container and access its elements. The iterator pattern decouples the iteration algorithms from container data structures.

Iterators take responsibility for two main actions:

1. Returning the data from a stream or container one item at a time
2. Keeping track of the current and visited items

In summary, an iterator will yield each item or value from a collection or a stream of data while doing all the internal bookkeeping required to maintain the state of the iteration process.

Python iterators must implement a well-established internal structure known as the iterator protocol. In the following section, you’ll learn the basics of this protocol.

What Is the Python Iterator Protocol?

A Python object is considered an iterator when it implements two special methods collectively known as the iterator protocol. These two methods make Python iterators work. So, if you want to create custom iterator classes, then you must implement the following methods:

The .__iter__() method of an iterator typically returns self, which holds a reference to the current object: the iterator itself. This method is straightforward to write and, most of the time, looks something like this:

def __iter__(self):
    return self

The only responsibility of .__iter__() is to return an iterator object. So, this method will typically just return self, which holds the current instance. Don’t forget that this instance must define a .__next__() method.

The .__next__() method will be a bit more complex depending on what you’re trying to do with your iterator. This method must return the next item from the data stream. It should also raise a StopIteration exception when no more items are available in the data stream. This exception will make the iteration finish. That’s right. Iterators use exceptions for control flow.

When to Use an Iterator in Python?

The most generic use case of a Python iterator is to allow iteration over a stream of data or a container data structure. Python uses iterators under the hood to support every operation that requires iteration, including for loops, comprehensions, iterable unpacking, and more. So, you’re constantly using iterators without being conscious of them.

In your day-to-day programming, iterators come in handy when you need to iterate over a dataset or data stream with an unknown or a huge number of items. This data can come from different sources, such as your local disk, a database, and a network.

In these situations, iterators allow you to process the datasets one item at a time without exhausting the memory resources of your system, which is one of the most attractive features of iterators.

Yielding the Original Data

Okay, now it’s time to learn how to write your own iterators in Python. As a first example, you’ll write a classic iterator called SequenceIterator. It’ll take a sequence data type as an argument and yield its items on demand.

Fire up your favorite code editor or IDE and create the following file:

# sequence_iter.py

class SequenceIterator:
    def __init__(self, sequence):
        self._sequence = sequence
        self._index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._index < len(self._sequence):
            item = self._sequence[self._index]
            self._index += 1
            return item
        else:
            raise StopIteration

Your SequenceIterator will take a sequence of values at instantiation time. The class initializer, .__init__(), takes care of creating the appropriate instance attributes, including the input sequence and an ._index attribute. You’ll use this latter attribute as a convenient way to walk through the sequence using indices.

The .__iter__() method does only one thing: returns the current object, self. In this case, self is the iterator itself, which implies it has a .__next__() method.

Finally, you have the .__next__() method. Inside it, you define a conditional statement to check if the current index is less than the number of items in the input sequences. This check allows you to stop the iteration when the data is over, in which case the else clause will raise a StopIteration exception.

In the if clause, you grab the current item from the original input sequence using its index. Then you increment the ._index attribute using an augmented assignment. This action allows you to move forward in the iteration while you keep track of the visited items. The final step is to return the current item.

Here’s how your iterator works when you use it in a for loop:

>>> from sequence_iter import SequenceIterator

>>> for item in SequenceIterator([1, 2, 3, 4]):
...     print(item)
...
1
2
3
4

Great! Your custom iterator works as expected. It takes a sequence as an argument and allows you to iterate over the original input data.

Before diving into another example, you’ll learn how Python’s for loops work internally. The following code simulates the complete process:

>>> sequence = SequenceIterator([1, 2, 3, 4])

>>> # Get an iterator over the data
>>> iterator = sequence.__iter__()
>>> while True:
...     try:
...         # Retrieve the next item
...         item = iterator.__next__()
...     except StopIteration:
...         break
...     else:
...         # The loop's code block goes here...
...         print(item)
...
1
2
3
4

After instantiating SequenceIterator, the code prepares the sequence object for iteration by calling its .__iter__() method. This method returns the actual iterator object. Then, the loop repeatedly calls .__next__() on the iterator to retrieve values from it.

When a call to .__next__() raises the StopIteration exception, you break out of the loop. In this example, the call to print() under the else clause of the try block represents the code block in a normal for loop. As you can see, the for loop construct is a kind of syntactic sugar for a piece of code like the one above.

Transforming the Input Data

Say that you want to write an iterator that takes a sequence of numbers, computes the square value of each number, and yields those values on demand. In this case, you can write the following class:

# square_iter.py

class SquareIterator:
    def __init__(self, sequence):
        self._sequence = sequence
        self._index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._index < len(self._sequence):
            square = self._sequence[self._index] ** 2
            self._index += 1
            return square
        else:
            raise StopIteration

The first part of this SquareIterator class is the same as your SequenceIterator class. The .__next__() method is also pretty similar. The only difference is that before returning the current item, the method computes its square value. This computation performs a transformation on each data point.

Here’s how the class will work in practice:

>>> from square_iter import SquareIterator

>>> for square in SquareIterator([1, 2, 3, 4, 5]):
...     print(square)
...
1
4
9
16
25

Using an instance of SquareIterator in a for loop allows you to iterate over the square values of your original input values.

The option of performing data transformation on a Python iterator is a great feature. It can make your code quite efficient in terms of memory consumption. Why? Well, imagine for a moment that iterators didn’t exist. In that case, if you wanted to iterate over the square values of your original data, then you’d need to create a new list to store the computed squares.

This new list would consume memory because it would have to store all the data simultaneously. Only then would you be able to iterate over the square values.

However, if you use an iterator, then your code will only require memory for a single item at a time. The iterator will compute the following items on demand without storing them in memory. In this regard, iterators are lazy objects.

Generating New Data

You can also create custom iterators that generate a stream of new data from a given computation without taking a data stream as input. Consider the following iterator, which produces Fibonacci numbers:

# fib_iter.py

class FibonacciIterator:
    def __init__(self, stop=10):
        self._stop = stop
        self._index = 0
        self._current = 0
        self._next = 1

    def __iter__(self):
        return self

    def __next__(self):
        if self._index < self._stop:
            self._index += 1
            fib_number = self._current
            self._current, self._next = (
                self._next,
                self._current + self._next,
            )
            return fib_number
        else:
            raise StopIteration

This iterator doesn’t take a data stream as input. Instead, it generates each item by performing a computation that yields values from the Fibonacci sequence. You do this computation inside the .__next__() method.

You start this method with a conditional that checks if the current sequence index hasn’t reached the ._stop value, in which case you increment the current index to control the iteration process. Then you compute the Fibonacci number that corresponds to the current index, returning the result to the caller of .__next__().

When ._index grows to the value of ._stop, you raise a StopIteration, which terminates the iteration process. Note that you should provide a stop value when you call the class constructor to create a new instance. The stop argument defaults to 10, meaning the class will generate ten Fibonacci numbers if you create an instance without arguments.

Here’s how you can use this FibonacciIterator class in your code:

>>> from fib_iter import FibonacciIterator

>>> for fib_number in FibonacciIterator():
...     print(fib_number)
...
0
1
1
2
3
5
8
13
21
34

Instead of yielding items from an existing data stream, your FibonacciIterator class computes every new value in real time, yielding values on demand. Note that in this example, you relied on the default number of Fibonacci numbers, which is 10.

Coding Potentially Infinite Iterators

An interesting feature of Python iterators is that they can handle potentially infinite data streams. Yes, you can create iterators that yield values without ever reaching an end! To do this, you just have to skip the StopIteration part.

For example, say that you want to create a new version of your FibonacciIterator class that can produce potentially infinite Fibonacci numbers. To do that, you just need to remove the StopIteraton and the condition that raises it:

# inf_fib.py

class FibonacciInfIterator:
    def __init__(self):
        self._index = 0
        self._current = 0
        self._next = 1

    def __iter__(self):
        return self

    def __next__(self):
        self._index += 1
        self._current, self._next = (self._next, self._current + self._next)
        return self._current

The most relevant detail in this example is that .__next__() never raises a StopIteration exception. This fact turns the instances of this class into potentially infinite iterators that would produce values forever if you used the class in a for loop.

To check if your FibonacciInfIterator works as expected, go ahead and run the following loop. But remember, it’ll be an infinite loop:

>>> from inf_fib import FibonacciInfIterator

>>> for fib_number in FibonacciInfIterator():
...     print(fib_number)
...
0
1
1
2
3
5
8
13
21
34
Traceback (most recent call last):
    ...
KeyboardInterrupt

When you run this loop in your Python interactive session, you’ll notice that the loop prints numbers from the Fibonacci sequence without stopping. To stop the loops, go ahead and press Ctrl+C.

As you’ve confirmed in this example, infinite iterators like FibonacciInfIterator will make for loops run endlessly. They’ll also cause functions that accept iterators—such as sum(), max(), and min()—to never return. So, be careful when using infinite iterators in your code, as you can make your code hang.

Inheriting From collections.abc.Iterator

The collections.abc module includes an abstract base class (ABC) called Iterator. You can use this ABC to create your custom iterators quickly. This class provides basic implementations for the .__iter__() method.

It also provides a .__subclasshook__() class method that ensures only classes implementing the iterator protocol will be considered subclasses of Iterator.

Check out the following example, in which you change your SequenceIterator class to use the Iterator ABC:

# sequence_iter.py

from collections.abc import Iterator

class SequenceIterator(Iterator):
    def __init__(self, sequence):
        self._sequence = sequence
        self._index = 0

    def __next__(self):
        if self._index < len(self._sequence):
            item = self._sequence[self._index]
            self._index += 1
            return item
        else:
            raise StopIteration

If you inherit from Iterator, then you don’t have to write an .__iter__() method because the superclass already provides one with the standard implementation. However, you do have to write your own .__next__() method because the parent class doesn’t provide a working implementation.

Here’s how this class works:

>>> from sequence_iter import SequenceIterator

>>> for number in SequenceIterator([1, 2, 3, 4]):
...     print(number)
...
1
2
3
4

As you can see, this new version of SequenceIterator works the same as your original version. However, this time you didn’t have to code the .__iter__() method. Your class inherited this method from Iterator.

The features inherited from the Iterator ABC are useful when you’re working with class hierarchies. They’ll take work off your plate and save you headaches.

Creating Generator Functions

To create a generator function, you must use the yield keyword to yield the values one by one. Here’s how you can write a generator function that returns an iterator that’s equivalent to your SequenceIterator class:

>>> def sequence_generator(sequence):
...     for item in sequence:
...         yield item
...

>>> sequence_generator([1, 2, 3, 4])
<generator object sequence_generator at 0x108cb6260>

>>> for number in sequence_generator([1, 2, 3, 4]):
...     print(number)
...
1
2
3
4

In sequence_generator(), you accept a sequence of values as an argument. Then you iterate over that sequence using a for loop. In each iteration, the loop yields the current item using the yield keyword. This logic is then packed into a generator iterator object, which automatically supports the iterator protocol.

You can use this iterator in a for loop as you would use a class-based iterator. Internally, the iterator will run the original loop, yielding items on demand until the loop consumes the input sequence, in which case the iterator will automatically raise a StopIteration exception.

Generator functions are a great tool for creating function-based iterators that save you a lot of work. You just have to write a function, which will often be less complex than a class-based iterator. If you compare sequence_generator() with its equivalent class-based iterator, SequenceIterator, then you’ll note a big difference between them. The function-based iterator is way simpler and more straightforward to write and understand.

Using Generator Expressions to Create Iterators

If you like generator functions, then you’ll love generator expressions. These are particular types of expressions that return generator iterators. The syntax of a generator expression is almost the same as that of a list comprehension. You only need to turn the square brackets ([]) into parentheses:

>>> [item for item in [1, 2, 3, 4]]  # List comprehension
[1, 2, 3, 4]

>>> (item for item in [1, 2, 3, 4])  # Generator expression
<generator object <genexpr> at 0x7f55962bef60>

>>> generator_expression = (item for item in [1, 2, 3, 4])
>>> for item in generator_expression:
...     print(item)
...
1
2
3
4

Wow! That was really neat! Your generator expression does the same as its equivalent generator function. The expression returns a generator iterator that yields values on demand. Generator expressions are an amazing tool that you’ll probably use a lot in your code.

Exploring Different Types of Generator Iterators

As you’ve already learned, classic iterators typically yield data from an existing iterable, such as a sequence or collection data structure. The examples in the above section show that generators can do just the same.

However, as their name suggests, generators can generate streams of data. To do this, generators may or may not take input data. Like class-based iterators, generators allow you to:

• Yield the input data as is
• Transform the input and yield a stream of transformed data
• Generate a new stream of data out of a known computation

To illustrate the second use case, check out how you can write an iterator of square values using a generator function:

>>> def square_generator(sequence):
...     for item in sequence:
...         yield item**2
...

>>> for square in square_generator([1, 2, 3, 4, 5]):
...     print(square)
...
1
4
9
16
25

This square_generator() function takes a sequence and computes the square value of each of its items. The yield statement yields square items on demand. When you call the function, you get a generator iterator that generates square values from the original input data.

Alternatively, generators can just generate the data by performing some computation without the need for input data. That was the case with your FibonacciIterator iterator, which you can write as a generator function like the following:

>>> def fibonacci_generator(stop=10):
...     current_fib, next_fib = 0, 1
...     for _ in range(0, stop):
...         fib_number = current_fib
...         current_fib, next_fib = (
...            next_fib, current_fib + next_fib
...         )
...         yield fib_number
...

>>> list(fibonacci_generator())
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
>>> list(fibonacci_generator(15))
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377]

This functional version of your FibonacciIterator class works as expected, producing Fibonacci numbers on demand. Note how you’ve simplified the code by turning your iterator class into a generator function. Finally, to display the actual data, you’ve called list() with the iterator as an argument. These calls implicitly consume the iterators, returning lists of numbers.

In this example, when the loop finishes, the generator iterator automatically raises a StopIteration exception. If you want total control over this process, then you can terminate the iteration yourself by using an explicit return statement:

def fibonacci_generator(stop=10):
    current_fib, next_fib = 0, 1
    index = 0
    while True:
        if index == stop:
            return
        index += 1
        fib_number = current_fib
        current_fib, next_fib = next_fib, current_fib + next_fib
        yield fib_number

In this version of fibonacci_generator(), you use a while loop to perform the iteration. The loop checks the index in every iteration and returns when the index has reached the stop value.

You’ll use a return statement inside a generator function to explicitly indicate that the generator is done. The return statement will make the generator raise a StopIteration.

Just like class-based iterators, generators are useful when you need to process a large amount of data, including infinite data streams. You can pause and resume generators, and you can iterate over them. They generate items on demand, so they’re also lazy. Both iterators and generators are pretty efficient in terms of memory usage. You’ll learn more about this feature in the following section.

Returning Iterators Instead of Container Types

Regular functions and comprehensions typically create a container type like a list or a dictionary to store the data that results from the function’s intended computation. All this data is stored in memory at the same time.

In contrast, iterators keep only one data item in memory at a time, generating the next items on demand or lazily. For example, get back to the square values generator. Instead of using a generator function that yields values on demand, you could’ve used a regular function like the following:

>>> def square_list(sequence):
...     squares = []
...     for item in sequence:
...         squares.append(item**2)
...     return squares
...

>>> numbers = [1, 2, 3, 4, 5]
>>> square_list(numbers)
[1, 4, 9, 16, 25]

In this example, you have two list objects: the original sequence of numbers and the list of square values that results from calling square_list(). In this case, the input data is fairly small. However, if you start with a huge list of values as input, then you’ll be using a large amount of memory to store the original and resulting lists.

In contrast, if you use a generator, then you’ll only need memory for the input list and for a single square value at a time.

Similarly, generator expressions are more memory-efficient than comprehensions. Comprehensions create container objects, while generator expressions return iterators that produce items when needed.

Another important memory-related difference between iterators, functions, data structures, and comprehensions is that iterators are the only way to process infinite data streams. In this situation, you can’t use a function that creates a new container directly, because your input data is infinite, which will hang your execution.

Creating a Data Processing Pipeline With Generator Iterators

As mentioned before, you can use iterators and generators to build memory-efficient data processing pipelines. Your pipeline can consist of multiple separate generator functions performing a single transformation each.

For example, say you need to perform a bunch of mathematical tranformations on a sample of integer numbers. You may need to raise the values to the power of two or three, filter even and odd numbers, and finally convert the data into string objects. You also need your code to be flexible enough that you can decide which specific set of transformations you need to run.

Here’s your set of individual generator functions:

# math_pipeline.py

def to_square(numbers):
    return (number**2 for number in numbers)

def to_cube(numbers):
    return (number**3 for number in numbers)

def to_even(numbers):
    return (number for number in numbers if number % 2 == 0)

def to_odd(numbers):
    return (number for number in numbers if number % 2 != 0)

def to_string(numbers):
    return (str(number) for number in numbers)

All these functions take some sample data as their numbers argument. Each function performs a specific mathematical transformation on the input data and returns an iterator that produces transformed values on demand.

Here’s how you can combine some of these generator functions to create different data processing pipelines:

>>> import math_pipeline as mpl

>>> list(mpl.to_string(mpl.to_square(mpl.to_even(range(20)))))
['0', '4', '16', '36', '64', '100', '144', '196', '256', '324']

>>> list(mpl.to_string(mpl.to_cube(mpl.to_odd(range(20)))))
['1', '27', '125', '343', '729', '1331', '2197', '3375', '4913', '6859']

Your first pipeline takes some numbers, extracts the even ones, finds the square value of those even numbers, and finally converts each resulting value into a string object. Note how each function provides the required argument for the next function on the pipeline. The second pipeline works similarly.

The Iterable Protocol

The iterable protocol consists of a single special method that you already know from the section on the iterator protocol. The .__iter__() method fulfills the iterable protocol. This method must return an iterator object, which usually doesn’t coincide with self unless your iterable is also an iterator.

To quickly jump into an example of how the iterable protocol works, you’ll reuse the SequenceIterator class from previous sections. Here’s the implementation:

# sequence_iterable.py

from sequence_iter import SequenceIterator

class Iterable:
    def __init__(self, sequence):
        self.sequence = sequence

    def __iter__(self):
        return SequenceIterator(self.sequence)

In this example, your Iterable class takes a sequence of values as an argument. Then, you implement an .__iter__() method that returns an instance of SequenceIterator built with the input sequence. This class is ready for iteration:

>>> from sequence_iterable import Iterable

>>> for value in Iterable([1, 2, 3, 4]):
...     print(value)
...
1
2
3
4

The .__iter__() method is what makes an object iterable. Behind the scenes, the loop calls this method on the iterable to get an iterator object that guides the iteration process through its .__next__() method.

Note that iterables aren’t iterators on their own. So, you can’t use them as direct arguments to the next() function:

>>> numbers = Iterable([1, 2, 3, 4])
>>> next(numbers)
Traceback (most recent call last):
    ...
TypeError: 'Iterable' object is not an iterator

>>> letters = "ABCD"
>>> next(letters)
Traceback (most recent call last):
    ...
TypeError: 'str' object is not an iterator

>>> fruits = [
...     "apple",
...     "banana",
...     "orange",
...     "grape",
...     "lemon",
... ]
>>> next(fruits)
Traceback (most recent call last):
    ...
TypeError: 'list' object is not an iterator

You can’t pass an iterable directly to the next() function because, in most cases, iterables don’t implement the .__next__() method from the iterator protocol. This is intentional. Remember that the iterator pattern intends to decouple the iteration algorithm from data structures.

Note how both custom iterables and built-in iterables, such as strings and lists, fail to support next(). In all cases, you get a TypeError telling you that the object at hand isn’t an iterator.

If next() doesn’t work, then how can iterables work in for loops? Well, for loops always call the built-in iter() function to get an iterator out of the target stream of data. You’ll learn more about this function in the next section.

The Built-in iter() Function

From Python’s perspective, an iterable is an object that can be passed to the built-in iter() function to get an iterator from it. Internally, iter() falls back to calling .__iter__() on the target objects. So, if you want a quick way to determine whether an object is iterable, then use it as an argument to iter(). If you get an iterator back, then your object is iterable. If you get an error, then the object isn’t iterable:

>>> fruits = [
... "apple",
... "banana",
... "orange",
... "grape",
... "lemon",
... ]

>>> iter(fruits)
<list_iterator object at 0x105858760>

>>> iter(42)
Traceback (most recent call last):
    ...
TypeError: 'int' object is not iterable

When you pass an iterable object, like a list, as an argument to the built-in iter() function, you get an iterator for the object. In contrast, if you call iter() with an object that’s not iterable, like an integer number, then you get a TypeError exception.

The Built-in reversed() Function

Python’s built-in reversed() function allows you to create an iterator that yields the values of an input iterable in reverse order. Аs iter() falls back to calling .__iter__() on the underlying iterable, reversed() delegates on a special method called .__reverse__() that’s present in ordered built-in types, such as lists, tuples, and dictionaries.

Other ordered types, such as strings, also support reversed() even though they don’t implement a .__reverse__() method.

Here’s an example of how reversed() works:

>>> digits = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> reversed(digits)
<list_reverseiterator object at 0x1053ff490>

>>> list(reversed(digits))
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

In this example, you use reversed() to create an iterator object that yields values from the digits list in reverse order.

To do its job, reversed() falls back to calling .__reverse__() on the input iterable. If that iterable doesn’t implement .__reverse__(), then reversed() checks the existence of .__len__() and .__getitem___(index). If these methods are present, then reversed() uses them to iterate over the data sequentially. If none of these methods are present, then calling reversed() on such an object will fail.

The Sequence Protocol

Sequences are container data types that store items in consecutive order. Each item is quickly accessible through a zero-based index that reflects the item’s relative position in the sequence. You can use this index and the indexing operator ([]) to access individual items in the sequence:

>>> numbers = [1, 2, 3, 4]
>>> numbers[0]
1
>>> numbers[2]
3

Integer indices give you access to individual values in the underlying list of numbers. Note that the indices start from zero.

All built-in sequence data types—like lists, tuples, and strings—implement the sequence protocol, which consists of the following methods:

• .__getitem__(index) takes an integer index starting from zero and returns the items at that index in the underlying sequence. It raises an IndexError exception when the index is out of range.
• .__len__() returns the length of the sequence.

When you use an object that supports these two methods, Python internally calls .__getitem__() to retrieve each item sequentially and .__len__() to determine the end of the data. This means that you can use the object in a loop directly.

To check this internal behavior of Python, consider the following class, which implements a minimal stack data structure using a list to store the actual data:

# stack.py

class Stack:
    def __init__(self):
        self._items = []

    def push(self, item):
        self._items.append(item)

    def pop(self):
        try:
            return self._items.pop()
        except IndexError:
            raise IndexError("pop from an empty stack") from None

    def __getitem__(self, index):
        return self._items[index]

    def __len__(self):
        return len(self._items)

This Stack class provides the two core methods that you’ll typically find in a stack data structure. The push() method allows you to add items to the top of the stack, while the pop() method removes and returns items from the top of the stack. This way, you guarantee that your stack works according to the LIFO (last in, first out) principle.

The class also implements the Python sequence protocol. The .__getitem__() method returns the item at index from the underlying list object, ._items. Meanwhile, the .__len__() method returns the number of items in the stack using the built-in len() function.

These two methods make your Stack class iterable:

>>> from stack import Stack

>>> stack = Stack()
>>> stack.push(1)
>>> stack.push(2)
>>> stack.push(3)
>>> stack.push(4)

>>> iter(stack)
<iterator object at 0x104e9b460>

>>> for value in stack:
...     print(value)
...
1
2
3
4

You Stack class doesn’t have an .__iter__() method. However, Python is smart enough to build an iterator using .__getitem__() and .__len__(). So, your class supports iter() and iteration. You’ve created an iterable without formally implementing the iterable protocol.

Iterating Through Iterables With for Loops

Iterables shine in the context of iteration. They’re especially useful in definite iteration, which uses for loops to access the items in iterable objects. Python’s for loops are specially designed to traverse iterables. That’s why you can use iterables directly in this type of loop:

>>> for number in [1, 2, 3, 4]:
...     print(number)
...
1
2
3
4

Internally, the loop creates the required iterator object to control the iteration. This iterator keeps track of the item currently going through the loop. It also takes care of retrieving consecutive items from the iterable and finishing the iteration when the data is over.

Iterating Through Comprehensions

Python also allows you to use iterables in another kind of iteration known as comprehensions. Comprehensions work similarly to for loops but have a more compact syntax.

You can use comprehensions to create new lists, dictionaries, and sets from existing iterables of data. In all cases, the comprehension construct will iterate over the input data, transform it, and generate a new container data type.

For example, say that you want to process a list of numeric values and create a new list with cube values. In this case, you can use the following list comprehension to perform the data transformation:

>>> numbers = [1, 2, 3, 4]

>>> [number**3 for number in numbers]
[1, 8, 27, 64]

This list comprehension builds a new list of cube values from the original data in numbers. Note how the syntax of a comprehension resembles a for loop with the code block at the beginning of the construct.

You can also use comprehensions to process iterables conditionally. To do this, you can add one or more conditions at the end of the comprehension construct:

>>> numbers = [1, 2, 3, 4, 5, 6]

>>> [number for number in numbers if number % 2 == 0]
[2, 4, 6]

The condition at the end of this comprehension filters the input data, creating a new list with even numbers only.

Comprehensions are popular tools in Python. They provide a great way to process iterables of data quickly and concisely. To dive deeper into list comprehensions, check out When to Use a List Comprehension in Python.

Unpacking Iterables

In Python, you can use iterables in a type of operation known as iterable unpacking. Unpacking an iterable means assigning its values to a series of variables one by one. The variables must come as a tuple or list, and the number of variables must match the number of values in the iterable.

Iterable unpacking can help you write more concise, readable code. For example, you may find yourself doing something like this:

>>> numbers = [1, 2, 3, 4]

>>> one = numbers[0]
>>> two = numbers[1]
>>> three = numbers[2]
>>> four = numbers[3]

>>> one
1
>>> two
2
>>> three
3
>>> four
4

If this is your case, then go ahead and replace the series of individual assignments with a more readable iterable unpacking operation using a single, elegant assignment like the following:

>>> numbers = [1, 2, 3, 4]

>>> one, two, three, four = numbers

>>> one
1
>>> two
2
>>> three
3
>>> four
4

In this example, numbers is an iterable containing numeric data. The assignment unpacks the values in numbers into the four target variables. To do this, Python internally runs a quick loop over the iterable on the right-hand side to unpack its values into the target variables.

The above example shows the most common form of iterable unpacking in Python. The main condition for the example to work is that the number of variables must match the number of values in the iterable.