Understanding That the GIL Won't Save You

00:00 In the previous lesson, I showed you a race condition. In this lesson, I’ll explain why the GIL doesn’t necessarily stop them from happening.

00:08 I vaguely remember indicating that concurrency is hard. I’d like to emphasize something though. Concurrency is hard. You might think that the GIL, which gets in the way by locking things, would stop these kinds of problems from happening, but it doesn’t.

00:22 Even with a single line of Python, you can have a race condition.

00:26 Remember, Python compiles your script into bytecode, and your single line of Python very well might turn into multiple lines of bytecode. The GIL only locks things at the bytecode level, so even your single line of Python could result in the GIL being acquired and released multiple times.

00:43 And if that isn’t complicated enough, just where the locking happens has changed over time. How the GIL works is an implementation detail of the interpreter and not part of the language itself.

00:54 The implementation of the interpreter has changed over time, as has when the locking happens. And that’s without the new added complication of free threading, which is the process of getting rid of the GIL altogether.

01:07 Yep. Concurrency is hard. At the outset, I mentioned that this course was originally created in 2020 and that this is an updated version. The code on the screen is from the original race condition lesson written using Python 3.8.

01:22 Note that the lack of a sleep call. The idea was the same, updating a value with plus or minus one, but it used a single line of Python. Without the sleep, it was harder to get the race condition to trigger.

01:34 So instead, the code ran a lot more. I made the change 10,000 times and ran it over a thousand pieces of data instead of the 50 you saw in the previous lesson.

01:44 To demonstrate the fact that a single line of Python isn’t necessarily a single line of bytecode, let me show you the bytecode. counter += amount turns into six bytecode operations.

01:56 The LOAD_NAME calls fetch variables onto the stack. The INPLACE_ADD does the +=, then the STORE_NAME puts it back in counter.

02:05 The other stuff is just what’s needed to get the code going and return the result. This is why that single line of Python can be a race condition. I don’t know which of these operations can and can’t trigger the GIL, but I suspect the load and store ones might as they touch memory.

02:20 This is all well and good. The demo was a nice little example of a race condition. Then a few years went by and a comment popped up on the course. One of the students couldn’t get the race condition to happen anymore.

02:33 I dug in and, well, turns out that later versions of Python changed the bytecode.

02:39 This is the bytecode for Python 3.14. Not only has it changed, but something behind this changed as well when the interpreter acquired and released the GIL. With the Python 3.10 release, they changed when the GIL locks, and a side effect of that was my single line was actually being protected by the GIL.

02:59 That might sound like good news. Yay, no more race condition, but it’s bad news. The race condition is still there in the code, and now it only surfaces in some interpreters.

03:09 This is one of the reasons they’re moving slowly with free-threaded Python. Lots of code out there that relies on the GIL has a chance to break once it’s in free-threaded mode.

03:18 Something that you didn’t think was a race condition might actually be a race condition, and when the GIL goes away, it’ll surface.

03:27 So just what do you do about this? Well, the GIL isn’t the only lock. Python has locking primitives that you can use yourself to protect your shared resources.

03:36 There’s a bunch of them, and they’re beyond the scope of this lesson, but if you want to learn more, this course or tutorial can teach you all about locks, semaphores, and their cousins.

03:47 I’d like to tell you that asyncio solves the race condition problem. Unfortunately, it doesn’t either, but it is another way of programming concurrency in Python and is of interest.