The Default Python Implementation

00:00 This is probably not the best website to be making bold claims like this, but I’ll say it anyways: Python is not that special. Really, it’s not. At its core, it’s just a language and nothing else.

00:14 We often think of Python as this program that we can install and use to run Python code, but that’s not 100% accurate. Python is just like a reference manual that specifies all the rules of the language.

00:29 Just like the English language states that every sentence should end with a period, Python states that blocks of code should be denoted by indentation and keywords like if, while, and for can be used to control the flow of execution.

00:46 You can read the entire Python language specification at docs.python.org.

00:53 CPython is the program that actually runs the Python code you write. It knows about all the rules of Python, so it can read your code and execute it as you’d expect.

01:04 This happens when you run Python, passing in your .py file as a program argument. It should be noted that CPython and Cython are not the same thing.

01:16 CPython does two things. First, it compiles your Python code down into what’s called bytecode. This is basically a giant list of instructions that’s hard for us humans to read but very easy for the computer to. If you’ve ever seen a .pyc file or a __pycache__ file near your Python code, this is the bytecode that was generated.

01:41 The bytecode is then run inside what is called the Python Virtual Machine. This is a special execution environment that reads in each line of bytecode and tells your computer how to carry out the associated operation. At the end of it all, you have a fully-functioning program.

01:59 The process of compiling Python code into bytecode is pretty complicated and it’s not our focus here, but believe it or not, the CPython code that actually evaluates the bytecode in the virtual machine is pretty straightforward. It is quite literally a giant switch statement, which is like a Python conditional block—you know, if, elif, elif, and so on. Each line of bytecode, called the opcode, is read in and then the switch block checks to see what operation should be performed. For example, NOP or LOAD_FAST.

02:37 It’s important to note that there are other implementations of Python too. These use the same Python language specification, but they might compile and run the code a little differently. As a .NET developer, I really like IronPython because it compiles Python code down to Microsoft’s Common Language Runtime, instead of bytecode for the Python Virtual Machine.

02:59 This allows me to use Microsoft technologies like the .NET framework with Python code. Others exist for other platforms too, but CPython is by far the most popular.

03:12 CPython gets its name from the fact that it was written using the C programming language. This was done to give CPython the speed it needs to execute quickly and efficiently.

03:24 All the memory management algorithms and structures exist in the CPython code, all written in C. This is a huge code base, but for this course we just need to focus on one entity in that C code, and that is the PyObject.

03:41 PyObject is the grand-daddy of all objects. As you may know, every variable you create in Python, whether it be from a class you code or from a primitive type like int, is considered a Python object. That is, it is of type object. In computer memory, this object type is represented by PyObject.

04:04 A PyObject is made from a C struct, which is similar to a Python class but with only instance attributes and no methods. The PyObject has two attributes: ob_refcnt and ob_type. ob_refcnt stores how many variables are storing, or referencing, this object.

04:25 ob_type stores information about what type of object this is, whether it be int, or the list, or one you create yourself by instantiating a Python class.

04:36 ob_refcnt is used for garbage collection. Once this number reaches 0, the garbage collector knows it can free whatever memory this PyObject is occupying. It does this by looking at ob_type, which stores information that tells the garbage collector how to free this memory. In this way, memory your program no longer needs is freed, and you didn’t have to do a single thing.

05:01 PyObject is just one example of a data structure C uses to run your Python programs. There are many others, such as C arrays, which represent Python lists.

05:13 Now that you understand the basics of how CPython works, we’ll take a look at what happens when memory is not properly managed. Depending on where the program is running, it could mean this.