Parallel Processing With multiprocessing: Overview
In this section, you’ll learn how to do parallel programming in Python using functional programming principles and the
multiprocessing module. You’ll take the example data set based on an immutable data structure that you previously transformed using the built-in
map() function. But this time you’ll process the data in parallel, across multiple CPU cores using the Python
multiprocessing module available in the standard library.
You’ll see, step by step, how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores. You’ll also learn how to use the
multiprocessing.Pool class and its parallel
map implementation that makes parallelizing most Python code that’s written in a functional style a breeze.
First, you’ll build a little testbed program that we can use to measure the execution time with the
time.time() function, so that we can compare the single-threaded and multithreaded implementations of the same algorithm.
00:00 Hey, and welcome to the next video in my Functional Programming in Python series. In this video, we’re going to talk about parallel programming: how can you execute code and do data processing in parallel using Python and
00:15 using functional programming principles. In this video, we’re going to take everything you’ve learned so far and use these techniques to very easily take some existing code and make it parallel so that it runs on all of your CPU cores at the same time, potentially reducing the amount of time it takes for your program to run.
Become a Member to join the conversation.