Generator Expressions
Generator expressions compute values lazily, producing items one at a time instead of building a full container in memory. They’re ideal when you only need to iterate once, or when the input data is large or streaming.
For generator expressions, these best practices help you balance concision and readability:
- Use generator expressions for one-pass transformations. Prefer generator expressions when you don’t need to keep the full result in memory and you’ll iterate over the values only once.
- Keep generator expressions flat. Avoid deeply nested generator expressions or ones with several conditions. If the logic becomes difficult to scan, switch to explicit loops or helper functions.
- Avoid side effects inside generator expressions. Like comprehensions, generator expressions work best for computing values—not for printing, writing files, or mutating external state.
- Use generator expressions for large or streaming data. When you don’t want all results in memory at once, generator expressions are typically more memory-efficient than building intermediate containers.
- Use generator expressions directly in function calls. Many built-in functions like
sum(),any(), andall()accept generator expressions, which can keep your code concise without creating intermediate lists. - Remember that generator expressions are single-use. Once exhausted, a generator expression can’t be reused. If you need to iterate multiple times, store the results in a concrete container instead.
To see some of these ideas in practice, check the following example that scans a log file to see whether it contains any error lines:
🔴 Avoid this:
>>> with open("app.log", encoding="utf-8") as file:
... error_lines = [line for line in file if "ERROR" in line]
...
>>> bool(error_lines)
True
This version works, but it reads the entire file and stores every matching line in memory.
✅ Favor this:
>>> with open("app.log", encoding="utf-8") as file:
... has_errors = any("ERROR" in line for line in file)
...
>>> has_errors
True
Here, you pass a generator expression directly to any(). Python checks each line lazily and stops as soon as it finds the first match. This approach avoids building an intermediate list and is especially memory-efficient on large files.
Related Resources
Tutorial
How to Use Generators and yield in Python
In this step-by-step tutorial, you'll learn about generators and yielding in Python. You'll create generator functions and generator expressions using multiple Python yield statements. You'll also learn how to build data pipelines that take advantage of these Pythonic tools.
For additional information on related topics, take a look at the following resources:
- Python Generators 101 (Course)
- How to Use Generators and yield in Python (Quiz)