Grouping Data With itertools.groupby()
Now that you know how to use the
reduce() function and Python’s
defaultdict class, which is defined in the
collections module, it’s time to look at some useful helpers in the
itertools module, such as
In the next section of this course, you’ll learn how to do parallel programming in Python using functional programming principles and the
multiprocessing module. You’ll start by taking the example data set based on an immutable data structure that you previously transformed using the built-in
map() function. But this time, you’ll process the data in parallel, across multiple CPU cores using the Python
multiprocessing module available in the standard library.
All right. I want to end this
reducer() example with another, well, arguably more Pythonic version of what we looked at previously. You can see, I played with this a bunch because well, this here is called
scientist_by_field5. I was basically trying to come up with ways to do this grouping in better and more readable ways.
00:22 Now, this is based on a dictionary expression and this kind of fits the theme that happened in the other videos in this series as well, where I showed you kind of the classical functional programming approach, and then showed you a more Pythonic version where we were often using list comprehensions or generator expressions to get to the same result, but kind of do it in a more Pythonic, more readable way.
I’m not sure if that’s the case here, like, I’m not sure if this is more readable, but you can do it. And there’s actually a helper function in Python that is the
itertools.groupby() function. It does stuff like that.
So, I mean, arguably, this is more Pythonic because it uses a dictionary comprehension, but I’m not sure if this reads much better. But, you know, it gets around the need for the
defaultdict. So, you know, I showed you a couple of ways to do it.
Sometimes it’s fun to sit down and spend some time to try and come up with, I guess, like, a single-line solution for this problem, but this is more like a fun exercise rather than something you should do in practice and in production code. But anyway, I hope this gave you a better idea of what the
reduce() function could be used for and maybe also some ideas on how it could be used in more creative ways to achieve that grouping, for example, and not just for the classical examples where, you know, you have this here, where we’re adding up a bunch of values and kind of boiling it down to a single integer, or something like that.
So, I hope we achieved that. I hope you learned a bunch of things about functional programming in Python here. And at this point, you should have a pretty good understanding of what functional programming is, what the
reduce() functions are—which are kind of the core primitives of functional programming—how they work in Python, and how you should probably not use them in Python, or
Become a Member to join the conversation.