Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Generators

In this lesson, you’ll learn about generators. Generators are a useful way to iterate through a sequence using constant memory. Here’s an example:

>>>
>>> g = (x for x in [1, 2, 3])
>>> g
<generator object>
>>> next(g)
1
>>> next(g)
2
>>> next(g)
3
>>> next(g)
StopIteration

You can also define generators using a function:

>>>
>>> def f():
...    yield 1
...    yield 2
...    yield 3
>>> f()
<generator object>
>>> g = f()
>>> next(g)
1
>>> next(g)
2
>>> next(g)
3
>>> next(g)
StopIteration

If you want to learn more, check out What Are Python Generators?

00:00 Generators are a special type of iterator that you can use to iterate over a sequence of values. They’re special because they’re lazily evaluated—that means that you only evaluate values when you need them.

00:11 Let’s create a generator to iterate over the values 0 through 5. So, g = and then it’s like a list comprehension, except you use parentheses so that this is a generator expression. (i for i in range(6))because we want to include 5, and range() is exclusive. g is a generator.

00:30 To get the next value, you type next(g). 0, 1, 2, 3, 4, 5. If you call next() again, it will raise a StopIteration because there are no more values to iterate through. What’s special about a generator is that once it’s done—as in, once we’ve exhausted all the values, you can’t get the values again—calling next() doesn’t give you anything. It will always raise an error.

00:54 So to get the values again, you have to instantiate it again, and then call next() again. Let’s compare this with a list. Let’s say we want to sum all the values between 0 and 1000, including 1000. Using a list, you would do sum(), list comprehension—[i for i in range(1, 1001)].

01:19 Let’s do the same thing with a generator. We’ll just replace these brackets with parentheses.

01:27 This is because sum(), under the hood, calls __iter__() on our generator, which actually just returns the generator, but I’ll save it into a variable called iterator. Then, it calls the next() over and over on our generator.

01:44 Same thing goes for the list.

01:51 So, why even use generators? Well, let’s look at how much memory is taken with a list comprehension. Let’s save this in a variable lst, import sys, and sys.getsizeof(lst).

02:04 So, this is 9,000 bytes. Let’s do this for our generator. I don’t remember if we made a g variable yet, but it’s fine, we’ll just redefine it.

02:15 Size of g is only 128 bytes. So you can imagine, as our list grows really large—millions and millions of values—this is going to take up a lot more memory. Generators, on the other hand, will always be this constant number of bytes.

02:32 So in an interview, if you’re asked to loop through something that might contain lots and lots of values, you should use a generator. You’d also want to use a generator if you don’t need to evaluate all the values at once—you just need to evaluate them one at a time.

02:46 So, another example might be—with our lst,

02:51 if you had, like, some slow_method(), or something, that you call with i on each time, this will evaluate the slow_method() for all the values at once and then proceed with the code, while a generator, like this, would only evaluate the slow_method() when you need to, like this.

03:08 There’s also something called a generator function, which is a function that when called returns a generator. Any function can be a generator function if it contains the word yield.

03:18 So, yield is like return, except when you call next() on this generator that is returned, it will resume where it left off. So, yield 1, yield 2, and yield 3.

03:28 So calling f(), you would think according to most rules would go inside the function, but this actually returns a generator object. To go inside, you have to call next() on that generator object, and you have to actually save this into a variable, and then call next(). 1, 2, 3.

03:46 Calling next() on f() like this is going to create a new generator each time. There are other examples of generator functions, and I will link a Real Python video on generators down below. In the next video, you’ll learn about dictionaries and defaultdicts, which are a very useful data structure to store default values.

James Uejio RP Team on April 27, 2020

If you want to learn more, here is a Real Python walkthrough video on generators: What Are Python Generators?

Become a Member to join the conversation.