Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set the default subtitles language in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Choosing a Queue

A queue is similar to a stack in that is an ordered collection of objects. The order is different though, it is First-In/First-Out (FIFO). This is like a line-up at a bank. For single threaded Python programs you can use either the built-in list type or collections.deque for a FIFO queue. For concurrent code, use either the queue.Queue or multiprocessing.Queue objects for multi-threaded or multi-processing, respectively.

Here are resources and additional documentation about queue, multiprocessing, concurrency, and pickle:

00:00 In the previous lesson, I talked about queues in threaded and multiprocessor environments. In this lesson, I’ll wrap up the section on queues and talk about how to choose a queue library.

00:12 The first factor in choosing an implementation of a queue is to determine whether or not you are doing concurrent code. If not, use the deque object in the collections library, unless you’re just doing very small amounts of data—then you can get away with a list.

00:27 If you are being brave and going down the whole concurrency road, your choice is bound by what kind of parallelism you are implementing. With threading, use the queue library. With multiprocessing, use the multiprocessing library.

00:41 That’s a pretty straightforward choice.

00:44 As always, a good source of info is the Python documentation. Here’s the collections library for the deque object, the thread-safe version of a queue, and the multiprocessing.Queue.

00:58 If you’re new to concurrency or coming from another programming language, understanding the GIL and its impact on parallel code in Python is important. This article can point you in the right direction.

01:09 If you’re using the multiprocessing.Queue, you’ll need to make sure whatever you’re sending between processes is pickleable. This article introduces you to Python’s pickle.

01:19 And if you’re after more understanding of threads, async I/O, multiprocessing, and how to use them to get faster, more responsive code, this course introduces all those concepts.

01:32 That’s all for FIFO queues. Next up, priority queues.

Become a Member to join the conversation.