Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Python Generators 101 (Summary)
In this course, you’ve learned about generator functions and generator expressions.
You now know:
- How to use and write generator functions and generator expressions
- How the all-important Python
yieldstatement enables generators - How to use multiple Python
yieldstatements in a generator function - How to use
.send()to send data to a generator - How to use
.throw()to raise generator exceptions - How to use
.close()to stop a generator’s iteration - How to build a generator pipeline to efficiently process large CSV files
How have generators helped you in your work or projects? If you’re just learning about them, then how do you plan to use them in the future? Did you find a good solution to the data pipeline problem?
Congratulations, you made it to the end of the course! What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment in the discussion section and let us know.
00:00 Welcome to our very last video. This is the concluding video. The first thing I’d like to do as we wrap up is to revisit the slide from the very first video. So, on this slide, I told you what you were going to learn in this course and I guess you could say I made a promise, so here we can check if I kept my word. We went over what generators are and how you would use them.
00:20 So, they’re basically structures which allow you to iterate and they return one value at a time, or rather, they yield one value at a time. And that’s quite nice because it saves a lot of memory.
00:33 Other iterator structures load everything into memory at once and so they have, or can have, a very significant memory footprint. There’s a tradeoff here and that is that generators save a lot of memory, but they do this at the expense of speed.
00:46 So whether or not they’re a good solution for you depends, really, on your use case and what it is that you’re trying to achieve.
00:52
I showed you how to create generator functions, the main thing here is to use the keyword yield, and I showed you how to use generator expressions.
01:01
They’re basically like list comprehensions, except they use parentheses instead of square brackets. We looked at the Python yield statement.
01:08
This is what makes a generator function a generator function, as opposed to just a normal function, and you use them by using the yield keyword instead of the return keyword. And yield actually has two uses, we’ll get back to that in a moment. You can use multiple yield statements in a single function. When this generator function is called, it will run until the next yield statement, and that’s where it’ll stop and it’ll wait. And finally, we looked at some advanced generator methods, again, we’ll go back to those in just a second. In the video just before this one, we looked at an example of a data pipeline that uses generators.
01:46 I would encourage you to try that code out by yourself. It’s on the website. You can copy-paste it and then play around with it, that’s really the best way to get the insights from that example to sink in.
01:58
Take it apart, try to understand how it works, try to break it, try to modify it, you know, get creative with it. Okay. So, here is a quick wrap up of the syntax. Generator functions look a lot like functions, except that they use yield instead of return.
02:14
You can have several yield statements. Generator expressions look a lot like list comprehensions, except that you use parentheses instead of square brackets.
02:24 This is easy to remember if you keep in mind that square brackets are used when you define lists. So, lists comprehensions—square brackets, generator expressions—parentheses.
02:36
The three advanced generator methods we saw were send(), throw(), close(). send() is sort of the trickiest one. send() allows you to send a value into the generator, and this is received at the spot where the generator stopped, so where the yield keyword is used, and it can be used to populate a variable within that generator.
02:56
Just keep in mind that this only works if the generator has been initialized first, so you can’t just do this on an unstarted generator. throw() allows you to catch exceptions and select the error type and the error message, which will be returned.
03:11
And finally, close() allows you to close a generator so it doesn’t float around and take up space. Okay, so that was it for this video course. I hope you learned a lot, I hope you enjoyed yourself.
03:22 There’s always a lot to discover, a lot to learn on the website. There are countless tutorials, videos, articles. There’s always something exciting, so I encourage you to stop by.
Super on June 17, 2020
This is really good. Thanks :-)
John Berliner on June 17, 2020
Very helpful. The flat file parser is a really interesting and useful example.
drawdoowmij on June 17, 2020
Really nice introduction to generators. Thanks!!
Alan ODannel on June 18, 2020
Very good introduction to generators. Nice examples as well. Thank you.
avalidzy on June 21, 2020
Excellent presentation! Would watch it again. Will find a way to use generators with Django. Thanks!
Sandivya Saxena on June 21, 2020
Very informative!
mikesult on June 25, 2020
Thanks Christian. I learned a lot. Nice examples of using generators and I especially liked the example of chaining them together. Also I liked the exploration in the REPL and the tip of using sys.getsizeof(something) and cProfile.run(‘something()’) for getting memory and performance info.
sunflower761 on Aug. 15, 2020
This was very helpful. Thank you!
Sam Martin on Oct. 4, 2020
Great video! Thank you.
Ghani on Oct. 14, 2020
Excellent course; I learned a lot, thank you.
Peter Kulik on Feb. 10, 2021
Thank you, this course is really helpful!
Andy Pickett on Feb. 10, 2021
Fantastic explanation thank you! Really helped with a topic I’ve been struggling with in another course.
shachi on May 11, 2021
Very nicely done. Thank you.
Konstantinos on Sept. 8, 2022
Questions made in “Using Advanced Generator Methods” were not answered by an RP Team member.
thehitman on Jan. 23, 2024
In the example of .send(), can you target the .send() to a specific yield when you have multiple yields?
I recently used a generator with the next() call without the yield statement:
# Finding the quarter number for the given month
quarter_number = next( \
(quarter for start_month,middle_month, end_month, \
quarter in quarter_data_tuples if month in \
[start_month,middle_month, end_month]), \
None) # When completed it returns None
# it allowed to test whether a valid "quarter_number"
# was returned or not (None)
Bartosz Zaczyński RP Team on Jan. 24, 2024
@thehitman
In the example of
.send(), can you target the.send()to a specificyieldwhen you have multiple yields?
Not directly. Sending a value into a running generator resumes its execution at the last yield statement that paused the generator.
The code you posted above is an example of a generator expression, which defines a generator object in place. Since it’s an expression, you can’t use the yield statement in it even if you wanted.
thehitman on Jan. 24, 2024
Thanks for your explanation. I am new to Python.
Become a Member to join the conversation.
davevikram on June 16, 2020
This is my first exposure to Generators. I found easy to follow with the samples provided.