Exploring Changes & Challenges With Concurrency

00:00 In the previous lesson, I showed you some patterns of concurrent computing. In this lesson, I’ll talk about some of the challenges that are caused by concurrency and what has been changing in Python’s approach.

00:11 Programming concurrent systems introduces all sorts of new challenges to your software. Here are some of the things to consider. You must make sure that you’ve got execution coordination.

00:21 Different processes have to be able to sync up with each other. Think of the image processing example I gave in the previous lesson. Let’s say your image was a megabyte large and split into 10,000 kilobyte chunks.

00:33 Each worker could work on one of those 10 kilobyte chunks, but in all likelihood, the borders between those chunks might impact the calculation of the chunk next door.

00:42 So although for the most part, the workers can work independently, when they reach the borders of the portion they’re working on, they need to speak with others.

00:50 Once you have multiple processes going, the operating system or the programmer themselves has to determine how memory is allocated across those processes.

00:59 This is hard enough in a regular program. It just becomes more complicated when dealing with concurrency. Scheduling is about when to determine which processes are active.

01:09 Thankfully, operating systems have done a lot of work on this. For the most part, when you’re writing concurrent programs, you just let the OS do the scheduling, but of course, that also means you’re giving up control, so there may be situations where that’s not ideal.

01:23 Typically, the reason you’re writing a concurrent application in the first place is to be able to get things done faster. And really what you’re talking about here is a higher throughput, more work done per unit time.

01:35 How you manage execution, coordination, memory allocation, and scheduling can change the throughput on your system, and you might need to fine-tune how these things behave in order to actually see speed up.

01:47 Somewhat related to scheduling is distribution. How do you split up the different concurrent pieces amongst the CPUs on your machine or between machines?

01:56 Writing concurrent software introduces algorithmic challenges.

02:01 Deadlocks are when two or more components are waiting on each other. If A is waiting on B to do something, and B is waiting for A, well, then nothing is ever going to happen.

02:11 And finally, resource starvation may also be a problem. The different components of your program may be fighting for memory, disk space, or access to the CPU.

02:22 Concurrency can be applied at different levels in Python. The simplest is at the operating system level. This means as a programmer, you don’t have to really worry about it.

02:31 You can run Python in a window or in your IDE and let the operating system take care of the fact that other things are happening at the same time.

02:39 If you have multiple processors available on your computer, you can use the multiprocessing library in Python to have code execute on those different processors.

02:48 Modern operating systems have a lighter weight way of getting concurrency called threads. Each thread in a program can execute different code. Threads can be scheduled to take advantage of the I/O-bound nature of your program.

03:02 In addition to having threads in the standard library, Python has another mechanism using coroutines called asyncio. And then recently added in Python 3.14 is a compromise between threads and multiprocessing, the ability to run multiple independent interpreters within the same Python process.

03:20 The rest of this course describes the asyncio, threading, and multiprocessing standard libraries in Python, and the differences between them.

03:30 When considering concurrency inside of Python, you have to be aware of the GIL. That’s the Global Interpreter Lock. The GIL is a thread lock or a mutex that makes sure that only one thread controls memory management in the interpreter at a time.

03:44 This limits the way multiple threads can operate inside of Python. When two concurrent parts of a program operate at the same time, you can get race conditions.

03:53 If you have this kind of race condition inside of memory allocation, you will get memory leaks. The GIL was introduced to solve this problem. One of the powers of Python is the ability to extend it using C.

04:05 You can write code in C and plug it in underneath, and then have it called by Python. This makes Python quite powerful, but it also makes memory allocation challenging.

04:15 The GIL has been a source of contention for a long time in Python. There have been several attempts to remove it over the years, but the latest actually looks like it might work.

04:23 Python 3.13 added free-threaded mode experimentally. While Python 3.14 has promoted it and built it into CPython. It still isn’t on by default, and the steering committee reserves the right to yank it back out if it turns out to cause too many performance issues with single-threaded code.

04:41 To be clear, the GIL is an implementation detail of the interpreter. CPython and PyPy both have it, but Jython and IronPython do not have a GIL. So depending on your underlying interpreter, you may or may not run into this problem.

04:55 When you run Python, it compiles your script into bytecode. Then the interpreter is what executes that bytecode. For a long time, CPython has supported having more than one of these interpreters running in the same process.

05:08 These are known as subinterpreters. Until Python 3.14, the subinterpreters were only available to C extensions, meaning they didn’t tend to get used a lot.

05:18 Python 3.14 has now added them at the language level. Subinterpreters are essentially independent of each other, similar to running multiple processes, but they’re lighter weight because they live within this same process.

05:30 And as each subinterpreter has its own GIL, that means one subinterpreter won’t be waiting for another’s memory management. The GIL’s still there, so depending on what you’re doing, the subinterpreter might or might not help your restrictions, but it’s a step.

05:45 Although the GIL is a challenge, the I/O-bound nature of most programs means that you can do concurrency in Python even with it in place. In the next lesson, I’ll introduce you to the threading library.

05:55 A good way to do just that.