Using Advanced Generator Methods
In this lesson, you’ll learn about the advanced generators methods of .send()
, .throw()
, and .close()
. To practice with these new methods, you’re going to build a program that can make use of each of the three methods.
As you follow along in the lesson, you’ll learn that yield
is an expression, rather than a statement. You can use it as a statement, but you can manipulate a yielded value. You are allowed to .send()
a new value back to the generator. You’ll also handle exceptions with .throw()
and stop the generator after a given amount of digits with .close()
.
00:00 After telling you about the basic syntax of generators and why and how you would use them in the last video, I think we’re ready to move on to some more advanced generator methods.
00:10
There are three of these, which you should really be aware of: send()
, throw()
, and close()
. I’m going to illustrate them using some sample code.
00:19 Let’s switch over to the IDE. So here we are, and there is a function here, which detects palindromes. A palindrome is a number or a word which is read the same way forwards and backwards.
00:31 So, examples of number palindromes could be 121, 111, 141, 747, and so on. Examples of words are things like level, radar, and so on. Okay. So, you don’t need to understand how this function works exactly in detail, just at a very high level. Here we’re checking if an input number has a single digit and if it does, then we skip it.
00:58
I guess we could argue whether or not a single digit is a palindrome, but they’re kind of special cases. Then here on lines 5 to 9, we are actually reversing the number and checking if the number itself is the same as its reversed version. If that is the case, we will return True
. If not, we will return False
. Okay.
01:22
This is how we are going to make use of our function. So, I’ll explain this in steps. At the top, you have the infinite_palindromes()
generator.
01:32
And how can you tell this is a generator? If you answered, “Because there’s a yield
keyword,” then you’re right. This is a function with a yield
keyword, and therefore, it’s a generator.
01:44
Let’s step through this. We start by setting a variable num
to 0
and then, while True:
—so this is an infinite loop—
01:53
we’re giving num
to is_palindrome()
and checking whether or not it’s a palindrome. If it is, it gets yielded back to us, remember? This is returned, and infinite_palindrome()
keeps state.
02:05
Now, this line here is quite interesting because you can see that yield
is to the right of an assignment operator. This works together with the code down here, so here we’re starting our infinite_palindromes()
generator,
02:20
we’re iterating through it, we’re printing out numbers which turn out to be palindromes. Then, if they turn out to be palindromes, we are extracting their length, so the number of digits in the palindrome, and here we’re adding a leading 1
to it.
02:36
So if 121
is a palindrome, then we will start our next search from 1000.
02:42
Because there are three digits in 121
, we’re putting 1
in front of that, that gives 1000
. What’s interesting here is the send()
method.
02:50 This is the first of the advanced methods I wanted to show you. What this is doing is this is sending this to our generator, and the way it does that is—remember we started the generator here—it ran until it found one palindrome.
03:06
When it’s done that, we extract the length of the palindrome, put a 1
in front of it, so for a three-digit palindrome, that’s three 0
’s with a 1
in front of them, or 1000
.
03:15
And then this send()
is sending that value, so 1000
, here in the place of num
. And that’s how yield
, in this case, is playing a double role. On the one hand, it returns a value to us, or yields a value to us. But on the other hand, it works as a kind of placeholder where we can slide a new value into it.
03:35
So now i
has a value of 1000
, and that’s the value which is then fed into the palindrome detector, and from where we will start looking for the next palindrome in the sequence.
03:46 Let’s try running this. I’m going to make the terminal a bit bigger so it’s easier to see.
03:54 And there you can see it’s giving us palindromes and each time it’s finding a palindrome which is one digit longer than the previous one. At first it’s quite quick, and then the sequences become longer and longer, so you can see that it starts to slow down, but it’s still running.
04:09
If we give it a few seconds, it will give us another palindrome, or we can just interrupt it and move on. Let’s do that. Okay. That was the send()
method.
04:18
The send()
method allows you to slip a value into the last yield
statement where you stopped and, keep in mind, you can have more than one yield
statement.
04:27
In our example, we only had one. The next thing we’re going to look at is the throw()
method. What throw()
allows us to do is to throw an exception in a generator.
04:37 I’ll show you how this works in practice. Let’s go back to the IDE. Let’s enlarge the code editing part again so that this is easier to read.
04:47
Now, remember, here in digits
we are extracting the length of the palindrome which we just found. I’m going to modify this a bit. And what I’m going to do is I’m going to add a check here, whether or not we have found a palindrome which has five digits.
05:02
So 121
would have three digits, 1221
would have four, and I guess 12421
would have five. And then that would trigger this line because this would be True
and it would execute this method.
05:17
So, what this is doing is that the throw()
method is causing the generator to throw an exception and we’re specifying which exception we want. So I’m telling it I want a ValueError
and I want the error message to read, "We don't like large palindromes"
. I’m going to save this, I will enlarge the terminal again, and I’m going to run it. Well, here at the terminal first.
05:43
And you can see I got a first palindrome 11
, then 111
, then 1111
, then 10101
has five digits, and we got this error message.
05:57
The next and final advanced method I’d like to talk about is the close()
method. As the name suggests, this allows us to close a generator.
06:06
So remember that generators keep state between each time they’re being called and close()
causes the generator to lose state. It sort of evaporates the way a normal function would if it hit a return
statement. So coming back to the IDE, if you’ll remember from a minute ago, I had introduced this throw()
method here.
06:25
I’m going to replace this with just a simple close()
, Save,
06:31 and then watch. When I run this, instead of giving us an exception, the generator will close. I’m going to clear the terminal and then run this.
06:40
And you can see that the generator has given us a StopIteration
exception, which is what normally happens when a generator is exhausted. So close()
caused the generator to stop and behave as if it had exhausted its sequence.
06:54
So those were the three advanced generator methods that I wanted to show you. First, there’s send()
. send()
allows you to send a value into where the generator last stopped, so where the yield
statement was.
07:06
Something to keep in mind is that you need to have reached a yield
statement, so the generator needs to be initialized before you can send something to it.
07:14
Then, throw()
allows you to cause the generator to throw an exception. You can define which error type you want and which error message you want.
07:22
Finally, close()
allows you to close a generator. It behaves by returning a StopIteration
exception. So, it behaves as if it had exhausted its sequence, which it’s generating.
07:33 Okay, that’s it for this video. In the next video, I’m going to be showing you how to create a data pipeline using generators. I’ll see you there!
zell0ss on June 17, 2020
Ahhh! forgot markdown: i mean we send 10 ** (digits)
and I changed to 10 ** (digits) -1
Pierre on June 19, 2020
Goodness, that was wonderfully clear. It’s the first time I’ve been able to follow a presentation of generators. Thanks, Christian.
My only regret is you didn’t walk through the calling of a generator comprehension as you did a generator function in video 2. I’ll do so myself expecting they are called in the same manner.
Thanks, again
Howard M Sherman on June 28, 2020
Something doesn’t make sense to me in the sequence of palindromes being printed. Using your .throw()
method sample, the output is:
11
111
1111
10101
Traceback (most recent call last):
.
.
ValueError: We don't like large palindromes
Following the logic of your code, I would think the sequence of palindromes printed would be:
11
101
1001
10001
Since the for loop is running pal_gen.send(10 ** (digits))
, it would seem to me first palindrome following each 10 ** (digits)
sent would be that number + 1. Am I missing something?
Eron on June 29, 2020
It’s nice, but I have a question: why don’t we raise an exception directly in test code or in the infinite_palindrome method instead using throw() method?
Imtiaz on July 24, 2020
I do not understand this statement in “infinite_palindromes” function.
i = (yield num)
num is yielding but yet in parentheses (generator expression)
Also, this whole loop is unclear to me. Can anyone explain or give me some pointer.
while True:
if is_palindrome(num):
i = (yield num)
if i is not None:
num = i
num += 1
el-jose on Aug. 9, 2020
Hey everyone
Same as @zell0ss and @Howard M Sherman I didn’t understand why the code was not printing 11, 101, 1001 … so, doing some research and debugging this is what I found:
- In the first loop, you’re yielding an object (num = 11), then you save the “result” of that yield operation, which is None (i == None), and then you yield that None. In the second loop, you yield an object (num = 101), then you save the “result” of that yield operation, but in this case you NEVER yield that result, so None does not appear in the output. in the third loop (num = 111) is happening the same that happens in the first one, this is, yielding the None result.
- the way that I found to fix it is to add another yield statement. this yield statement is invoked only in the even loops, in order to yield the “None” result and print the correct number:
def infinite_palindromes():
"""
Palindromes generator
"""
num = 0
while True:
if is_palindrome(num):
i = yield num
yield num # this is the new line
if i is not None:
num = i
num += 1
also I added a time delay in the for cycle to actually see the result because for some reason, works faster than the original code:
for i in gen_pal:
time.sleep(1)
print(i)
digits = len(str(i))
gen_pal.send(10 ** (digits))
Hopefully all this explanation can be helpful :)
loopology on Nov. 29, 2020
It’s frustrating when it takes longer to understand the example than the concept being explained.
Why are we skipping to the next longer palindrome when we found the first for the current digit count? After 121 there are many more three-digit palindromes before we reach 1000. Why are we skipping them? Do we only want the first palindrome per digit count? If so, this deserves explicit mention.
Also, it’d be worth defining what exactly comprises a palindrome: According to the usual definition (“reads the same forward as backwards”), a one-digit number is a palindrome. This definition has the advantage to lend itself to recursive algorithms to detect palindromes.
Finally, these types of constructs:
if num == reversed_num:
return True
else:
return False
are more succinct and easier to read:
return num == reversed_num
Jon David on Nov. 11, 2021
My inner Bash programmer asserted itself and I rewrote is_palindrome
as:
is_palindrome = lambda val: int(''.join(reversed([x for x in str(val)]))) == val
This does count single digits as palindromes, but I can’t help liking it more :))
Jon David on Nov. 11, 2021
I just got an even better lambda from the realpython slack group!
is_palindrome = lambda x: str(x) == str(x)[::-1]
elegance += 20
Konstantinos on Sept. 7, 2022
I cannot understand why i = (yield num)
for “num = 101” or “num = 1001”, sets i
to None
. It also seems that for these numbers, yield
does not return to the generator consumer process. It is not explained at all in the course, though it’s critical to understand generators.
dheerajbhosale96 on Oct. 1, 2022
I found this tutorial little difficult to grasp. Checkout this resource for easy reading and better examples
pranayteja72 on May 4, 2023
hi i’m trying to return the generator expression from one function to another
def infinite_seq(n):
k=(num**2 for num in range(n))
print(type(k)). ->generator
return k
def calling():
r=(infinite_seq(5))
print(type(r)) ->generator
n=list(r)
print(n)
c=calling()
here my question is how to view the r value in better way rather than converting it to list . note:when i’m trying to use r=next(infinite_seq(5)) it is printing 0
Become a Member to join the conversation.
zell0ss on June 17, 2020
Great tutorial,
A curiosity I seem unable to answer, why the palindromes found are:
11 111 1111 10101
instead
11 101 1001 10001
At first I thought that was because we send 10 ** (digits) and it got incremented just afterwards so it will skip 101, 1001, … I changed it to 10 ** (digits) -1 but got the same result.
I know it is not strictly generator related, but nonetheless, Im curious!