In this lesson, you’ll learn about using concurrency with Python stacks. To learn more about concurrency, check out Speed Up Your Python Program With Concurrency.
Concurrency With Python Stacks
00:10 There’s a great Real Python tutorial series on using concurrency in Python. And a quick disclaimer before I get into the guts of concurrency with Python stacks: if you don’t see the immediate need or you don’t think that you’re going to ever need to use multithreading or concurrency with Python stacks, you can safely skip this video and move on to the last video in the series.
00:33 But if you want to use concurrency with stacks and you understand concurrency already, then stick with me and I’ll run you through the best ways to do that with Python stacks. So, as you’ll know if you’re familiar with concurrency, what you really want in concurrent programming is thread safety.
00:49 That is, you want to be able to have your threads execute operations in a sensible order, not step on each other’s toes, and normally the way to do that is to assure that the functions you’re calling are atomic or as close to it as possible so that there’s not a chance between operations for other threads to come in and do work you don’t want them to do.
deque implementations of a stack that I talked about in the last lesson aren’t completely perfect for thread safety. Lists are generally not considered thread-safe at all, and that’s a little bit of a misnomer because some
list operations are in fact thread-safe, but you really have to have an incredibly thorough understanding of the implementation of
list to know in what circumstances which functions of the
list class are thread-safe.
So we generally don’t want to use
list to implement stacks in a threading context. Now,
deque along with its slight speed advantages over
list gives us some advantages in thread safety as well, because
.pop()—the two operations you use most in a stack—are in fact atomic.
The problem, though, is that not all functions in the
deque class are atomic. So if you’re, for example, working with many other programmers or maybe you’re just looking back at your own code several months after you originally wrote it or something like that, the new person looking at your code might not know the
deque module as well as you do, and they might not know that they can only safely use
02:18 So you’re going to have to do some more work with documentation, with training to make sure that you don’t use other functions of that class and compromise thread safety. So this can be very thread-safe, a very thread-safe implementation, but it requires real precision and clarity in what functions you can use.
And these are really the only two that you can. So to solve this problem, enter
queue.LifoQueue. And in the
queue module is a bunch of great queue- or list-based data structure implementations in Python. It’s a great module. And it has a class called
LifoQueue, and remember that a stack has this Last-In/First-Out order, so LIFO.
And so the compromise there is that you have a slight speed loss as compared to a
deque implementation. So if you really, really need speed above all else, you might want to use a
deque and just be careful with your documentation to make sure that no one uses other functions of the
But if you really need that airtight thread safety,
LifoQueue is the way to go. Let’s move into the terminal and take a look at the
LifoQueue in action, because it has slightly different syntax than the other two methods that I’ve covered so far.
In this implementation you say
.put() instead of
.push(). But it’s exactly the same thing—
3. And then, let’s see. If you look at this, it just gives you this
queue.LifoQueue object, but I believe—yes.
.get(), here we go,
stack.get(). Now, the interesting thing and the thing you really have to be careful with when you use this implementation of a stack is that
LifoQueue.get(), as it were—when called on an empty list will just hang infinitely if there’s nothing there. Because the
stack.get() method, when it’s a
LifoQueue, is waiting for other threads to put things on the stack in order to get them again. So it’s going to wait infinitely, because at the moment I only have one thread running, and so there’s nothing else that’s going to put anything on there for it to get. So instead, what you want to do if you want to get an error is use
.get_nowait(), which will get but it won’t wait for any other threads to add.
And in that case, you get the familiar
_queue.Empty stack error. So, this is a different implementation and one that’s really focused on thread safety, but that adds some additional nuance to your understanding of how it works.
And you need to be careful of doing things like this, because
.get() will wait, and so you might need to use
.get_nowait() or you might need to just adjust your programming approach. So. That’s
Become a Member to join the conversation.