Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Choosing a Queue

A queue is similar to a stack in that is an ordered collection of objects. The order is different though, it is First-In/First-Out (FIFO). This is like a line-up at a bank. For single threaded Python programs you can use either the built-in list type or collections.deque for a FIFO queue. For concurrent code, use either the queue.Queue or multiprocessing.Queue objects for multi-threaded or multi-processing, respectively.

Here are resources and additional documentation about queue, multiprocessing, concurrency, and pickle:

00:00 In the previous lesson, I talked about queues in threaded and multiprocessor environments. In this lesson, I’ll wrap up the section on queues and talk about how to choose a queue library.

00:12 The first factor in choosing an implementation of a queue is to determine whether or not you are doing concurrent code. If not, use the deque object in the collections library, unless you’re just doing very small amounts of data—then you can get away with a list.

00:27 If you are being brave and going down the whole concurrency road, your choice is bound by what kind of parallelism you are implementing. With threading, use the queue library. With multiprocessing, use the multiprocessing library.

00:41 That’s a pretty straightforward choice.

00:44 As always, a good source of info is the Python documentation. Here’s the collections library for the deque object, the thread-safe version of a queue, and the multiprocessing.Queue.

00:58 If you’re new to concurrency or coming from another programming language, understanding the GIL and its impact on parallel code in Python is important. This article can point you in the right direction.

01:09 If you’re using the multiprocessing.Queue, you’ll need to make sure whatever you’re sending between processes is pickleable. This article introduces you to Python’s pickle.

01:19 And if you’re after more understanding of threads, async I/O, multiprocessing, and how to use them to get faster, more responsive code, this course introduces all those concepts.

01:32 That’s all for FIFO queues. Next up, priority queues.

Become a Member to join the conversation.