Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

How to Create a multiprocessing.Pool() Object

Give Feedback

In this lesson, you’ll create a multiprocesing.Pool object. This is an interface that you can use to run your transform() function on your input data in parallel, spread out over multiple CPU cores. This Pool instance has a map() function, so you can map() the transform() function over scientists.

Now, when you run your program, you’ll see that you get the same result, but you get it a lot faster. This happened because you did your processing in two batches. In the next lesson, you’ll keep working with multiprocessing.Pool().

cdrr930725 on Dec. 1, 2019

When I run the code my terminal starts going crazy, and I never got the desire output. My code is a replica of the lecture code. Please check the link bellow:

codeshare.io/2B4mxy

cdrr930725 on Dec. 1, 2019

I fix it by putting all the code into if __name__ == '__main__':

linusblady on March 27, 2020

cdrr930725 thanks for the tip. With if name == ‘main’: it runs. If you use the standard IDLE the print statement in the function will not be printed.

dorellaurent on April 8, 2020

Hello, I’m on Window 7 os. I work with IDLE. When I run the script, nothing is printed in the IDLE shell window. I tried with the if name == ‘main‘ part and the issue was the same…

renatoamreis on April 27, 2020

I have exactly the same problem as reported above. I also tried with name == ‘main‘

Dan Bader RP Team on April 27, 2020

Quick update on running these examples with IDLE (or presumably also other REPL environments):

You’ll probably run into issues if you don’t run the examples with python your_script_name.py from the command line, like I do in the video.

There are known issues with multiprocessing and IDLE (see this StackOverflow discussion for example)

norcal618 on May 28, 2020

I was able to get the multiprocessing stuff to run by putting some of the code into a function as so…

def run():
    start = time.time()
    pool = multiprocessing.Pool()
    result = pool.map(transform, scientists)
    end = time.time()
    print(f"\nTime to complete: {end - start:.2f}\n")
    pprint(result)

if __name__ == '__main__':
    run()

And the remaining code is left as show in the video

Arif Zuhairi on Oct. 9, 2020

I think because Dan run with Mac and we run with Windows and terminal/cmd prompt going crazy.. Thank you for the fix

Lucy on Oct. 13, 2020

Hi. i’m following very close your comments and fixes, but still gettin this error when using multiprocessing:

pickle.PicklingError: Can’t pickle <class ‘main.Scientist’>: it’s not the same object as main.Scientist

I also tried the norcal618’s code, but same error. Need to understand what is happening to continue learning and advancing. thanks in advance for any help

Become a Member to join the conversation.