Parallel Processing With multiprocessing: Overview
In this section, you’ll learn how to do parallel programming in Python using functional programming principles and the
multiprocessing module. You’ll take the example data set based on an immutable data structure that you previously transformed using the built-in
map() function. But this time you’ll process the data in parallel, across multiple CPU cores using the Python
multiprocessing module available in the standard library.
You’ll see, step by step, how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores. You’ll also learn how to use the
multiprocessing.Pool class and its parallel
map implementation that makes parallelizing most Python code that’s written in a functional style a breeze.
First, you’ll build a little testbed program that we can use to measure the execution time with the
time.time() function, so that we can compare the single-threaded and multithreaded implementations of the same algorithm.
Comments & Discussion
Become a Member to join the conversation.