When Not to Use a List Comprehension in Python
00:09 For example, they might make your code run more slowly or use more memory. If your code is less performant or harder to understand, then it’s probably better to choose an alternative. Firstly, we need to watch out for nested comprehensions.
00:24 Comprehensions can be nested to create combinations of lists, dictionaries, and sets within a collection. For example, say a climate laboratory is tracking the high temperature in five different cities for the first week in June.
The perfect data structure for storing this data could be a Python list comprehension nested within a dictionary comprehension. You create the outer collection
temps with a dictionary comprehension.
Let’s take a look at another code block for this. The outer list comprehension
[... for _ in range(6)] creates six rows, while the inner list comprehension
[i for i in range(5)] fills each of these rows with values.
Take this example, which uses a nested list comprehension to flatten a matrix. The code to flatten the matrix is concise, but it may not be so intuitive to understand how it works. On the other hand, if you were to use
for loops to flatten the same matrix, then your code will be much more straightforward.
02:01 Now you can see that the code traverses one row of the matrix at a time, pulling out all the elements in that row before moving on to the next one. While the single-line list comprehension might seem more Pythonic, what’s most important is to write code that your team can easily understand and modify.
02:43 If you want to sum the squares of the first one thousand integers, then a list comprehension will solve this problem admirably. But what if you wanted to sum the squares of the first billion integers?
02:54 If you try it on your machine, you might notice that your computer becomes non-responsive. That’s because Python is trying to create a list with one billion integers, which consumes more memory than your computer would like. Your computer may not have the resources it needs to generate an enormous list and store it in memory.
03:13 If you do try to do it anyway, then your machine could slow down or even crash. When the size of a list becomes problematic, it’s often helpful to use a generator instead of a list comprehension in Python.
03:26 A generator doesn’t create a single large data structure in memory, but instead returns an iterable. Your code can ask for the next value from the iterable as many times as necessary or until you’ve reached the end of your sequence, while only storing a single value at a time.
map() function also operates lazily, meaning that memory won’t be an issue if you choose to use it in this case. Once again I’ve sped up the video, but this just ran for a few minutes on my machine before it completed.
05:00 So, which approach is faster? Should you use list comprehensions or one of their alternatives? Rather than adhere to a single rule that’s true in all cases, it’s more useful to ask yourself whether or not performance matters in your specific circumstance. If not, then it’s usually best to choose whatever approach leads to the cleanest code.
If you’re in a scenario where performance is important, then it’s typically best to profile different approaches and listen to the data.
timeit is a useful library for timing how long it takes chunks of code to run.
As the code demonstrates, the biggest difference is between the loop-based approach and
map(), with the loop taking 50% longer to execute. Whether or not this matters depends on the needs of your application. It’s up to you whether you prefer the generator expression or
Become a Member to join the conversation.