Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
The multiprocessing Module
In this lesson, you’ll see why you’d want to take this approach. Because you wrote your code in a functional programming style, you can parallelize it fairly easily. There’s a parallel map construct that you can use. That way, you can run your processing steps in parellel.
You’ll import the multiprocessing module because it has all the building blocks you’ll need to run this operation in parallel. Later, you’ll learn how to use the multiprocessing.Pool class and its parallel map implementation, which makes parallelizing most Python code that’s written in a functional style a breeze.
00:00 Now, you may be wondering, okay, “Why are we doing this?” right? Like, “This is kind of obvious, why do I even care about this?” So, the cool thing is because we wrote our code in a functional programming style, we can actually parallelize it very easily because there is a parallel map construct we can use. That way, we can run these calculations, these processing steps that now happen sequentially—meaning, one after another on a single CPU core—we can parallelize them and run many of them in parallel.
00:37
And now that we have all of this stuff in place, this is very, very easy to do in Python. This is what I’m going to show you in this video. So, what I’m going to do next is I’m going to import the multiprocessing
00:55 because that has all of the building blocks that we need to actually run this operation in parallel.
Become a Member to join the conversation.