Free Threading and JIT

00:00 In the previous lesson, I showed you the improvements to the REPL. In this lesson, I’ll be talking about the two new experimental performance features.

00:08 Python 3.13 includes two experimental features, free threading and a JIT.

00:15 I put quotes surrounding “includes” because they’re not there by default. These features are available when you’re building Python as optional flags. If you happen to be on a Mac or Windows, the installer does have options for these features under the customized installation screen.

00:31 If you’re not on a Mac or Windows, you’ll have to build your own interpreter to play with these. So just what are these experimental features? The important word in free threading is free, as in free from the GIL.

00:45 If you haven’t done much concurrent programming, you might not be aware of the GIL. That’s short for Global Interpreter Lock. It’s a lock that prevents race conditions when Python is doing memory management.

00:55 This lock happens deep within the interpreter so unless you’ve been doing concurrency or you’ve written a library that uses the Python extensions mechanism, you wouldn’t necessarily have come across it before.

01:07 The GIL is both good and bad. The good is it’s a very quick way of handling ugly management problems. The bad is it limits the amount of concurrency you can get on a single thread on a single CPU.

01:19 Other programming languages use alternative ways of handling the same problem, and as modern machines have very fast CPUs, lots of people want the GIL to go away to remove the limitation.

01:31 The free threading feature is the first step in possibly removing the GIL for good. Right now, the feature is experimental. It’s not in the main distribution.

01:41 After a few releases, it will become optional as a flag to the interpreter, and then eventually the GIL will be off by default. For backward compatibility, it will still be available in case your code depends on it.

01:56 All that stuff I just said may not be the way it works out. There has been an attempt to get rid of the GIL before, and the performance consequence was too high, and so the project got squashed.

02:07 As it stands, the plan is multi-year and likely will play out over the next four or five years.

02:12 One of Python’s superpowers is its ability to interact with lower-level languages. This is how you can get the speed of NumPy while still having a high level language like Python.

02:23 This interaction is done through the application binary interface. Essentially a mechanism for compiled code to interact with the interpreter. This kind of compiled code is often called a C-extension, even though C is no longer the only language used at the lower level.

02:39 Extensions need to understand how to interact with the GIL, and as such, for the next little while, anyone writing an extension will have to compile it twice, once for the GIL and once for free-threading mode.

02:52 The other experimental feature is the JIT, which is short for Just-In-Time compilation. Currently, Python compiles your code to bytecode. Bytecode is a binary format that the interpreter reads, and that’s how your programs get run.

03:08 It’s platform-independent, so there is a level of abstraction between your machine and the compiled code. What the JIT does is actually compile your code down to machine code, which in theory means your code should be faster.

03:21 I say in theory because compilation takes time, and so often the overhead of compilation is more expensive than the performance boost it gives.

03:30 To get around this, the new JIT uses a mechanism called copy-and-patch. This is a JIT compiler that uses pre-compiled chunks that are common patterns found in code.

03:41 What the JIT does is fill in your variable references and the like, which means compilation is pretty much just a file copy. The upside is speed, the downside is your optimization is only as good as the set of templates to copy from.

03:56 For the experimental release, the core developers aren’t even hoping to gain any speed. They’re happy with the JIT just not being worse than regular Python.

04:05 Once they’ve got the concept working properly, then they’ll come along and optimize the optimization, hopefully resulting in faster execution.

04:14 The last few Python releases have had improvements to error messages. Python 3.13 doesn’t have a lot, but the changes keep on coming.