Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Parallel Processing With concurrent.futures: Overview

Give Feedback

In this section, you’ll learn how to do multithreading and parallel programming in Python using functional programming principles and the concurrent.futures module.

You’ll take the example data set based on an immutable data structure that you previously transformed using the built-in map() function. But this time, you’ll process the data in parallel, across multiple threads using Python 3’s concurrent.futures module, which is available in the standard library.

You’ll see, step by step, how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores and computing power. You’ll learn how to use the ProcessPoolExecutor and ThreadPoolExecutor classes and their parallel map implementations that make parallelizing most Python code written in a functional style a breeze.

By knowing the difference between both executors available in the concurrent.futures module, you’ll be able to parallelize your Python functions across multiple threads and across multiple processes. You’ll get a brief introduction to the Python Global Interpreter Lock, also known as the GIL, and see how you can work around its limitations by using the correct executor implementation.

Once again, you’ll use your little testbed program from the last video to measure the execution time with the time.time() function. This allows you to compare the single-threaded and multithreaded implementations of the same algorithm.

Comments & Discussion

mikkoskilpelainen on July 7, 2020

Such great content about Multiprocessing module and how it relates to map. Keep it up!

Become a Member to join the conversation.