Cool New Features in Python 3.12

Python 3.12: Cool New Features for You to Try

by Geir Arne Hjelle Oct 02, 2023 intermediate python

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: What's New in Python 3.12

Python 3.12 was published on October 2, 2023. As usual, the new version comes out in October after lots of effort by volunteers worldwide.

The new version comes with several new features and improvements that you’ll explore in this tutorial. You can also dive into the documentation to see a complete list of all changes.

In this tutorial, you’ll learn about new features and improvements, like:

  • Better error messages with helpful suggestions and guidance
  • More expressive f-strings that are backed by Python’s PEG parser
  • Optimizations, including inlined comprehensions, for a faster Python
  • A new syntax for type variables that you use to annotate generics
  • Support for the powerful perf profiler on Linux

If you want to try any of the examples in this tutorial, then you’ll need to use Python 3.12. The tutorials Python 3 Installation & Setup Guide and How Can You Install a Pre-Release Version of Python? walk you through several options for adding a new version of Python to your system.

In addition to learning more about the new features coming to the language, you’ll also get some advice about what to consider before upgrading to the new version. Click the link below to download code examples demonstrating the new capabilities of Python 3.12:

Improved Error Messages

Python is usually recognized as a good beginner language, and it’s lauded for its readable syntax. One area where it’s become even more user-friendly lately is error messages.

In Python 3.10, many error messages—especially for syntax errors—got more informative and precise. Similarly, Python 3.11 added more information in tracebacks, making it more convenient to pinpoint offending code.

The latest version of Python continues the work of improving your developer experience by providing better error messages. In particular, several common error messages now come with helpful suggestions. In the rest of this section, you’ll explore the new and improved messages.

Several of the improvements relate to importing modules. In the next three examples, you try to work with π by importing pi from math. In each example, you’ll see one of the new suggestions in Python 3.12. Here’s the first one:

Python
>>> math.pi
Traceback (most recent call last):
  ...
NameError: name 'math' is not defined. Did you forget to import 'math'?

When you use math without importing it first, you’ll get a traditional NameError. Additionally, the parser helpfully reminds you that you need to import math before accessing it.

The reminder about remembering to import modules only triggers for standard library modules. For these error messages, Python 3.12 doesn’t track third-party packages that you’ve installed.

It’s possible to import specific names from a module using a from-import statement. If you happen to switch the order of the keywords, you’ll now get a friendly nudge towards the correct syntax:

Python
>>> import pi from math
  ...
    import pi from math
    ^^^^^^^^^^^^^^^^^^^
SyntaxError: Did you mean to use 'from ... import ...' instead?

Here, you tried to import pi from math, but Python needs you to reorder the statement and put from before import.

To see a third new error message, check out what happens if you import py and not pi from math:

Python
>>> from math import py
Traceback (most recent call last):
  ...
ImportError: cannot import name 'py' from 'math'. Did you mean: 'pi'?

There’s no py name in math, so you get an ImportError. The parser suggests that you probably meant pi instead of py. Python 3.10 introduced a similar suggestion feature, where Python looks for similar names. What’s new in Python 3.12 is the ability to do this while importing.

In addition to these three improvements related to imports, there’s a final improvement concerning methods defined inside classes. Have a look at the following implementation of a Circle class:

Python
 1# shapes.py
 2
 3from math import pi
 4
 5class Circle:
 6    def __init__(self, radius):
 7        self.radius = radius
 8
 9    def area(self):
10        return pi * radius**2

Note that you wrongly refer to radius instead of self.radius inside .area(). This will cause an error when you call the method:

Python
>>> from shapes import Circle
>>> Circle(5).area()
Traceback (most recent call last):
  ...
  File "/home/realpython/shapes.py", line 10, in area
    return pi * radius**2
                    ^^^^^^
NameError: name 'radius' is not defined. Did you mean: 'self.radius'?

Instead of raising a plain NameError, Python recognizes that .radius is an attribute available on self. It then suggests that you use the instance attribute self.radius instead of the local name radius.

The suggestions that you’ve seen in these examples are all new in Python 3.12. Together, they make Python a little more user-friendly. You can learn more about these error message improvements and how they’ve been implemented in Python 3.12 Preview: Ever Better Error Messages.

More Powerful F-Strings

Formatted strings, or f-strings for short, were introduced in PEP 498 and Python 3.6. With f-strings, Python added string interpolation to the language. You can recognize f-strings by the leading f in examples like the following:

Python
>>> from datetime import date
>>> major = 3
>>> minor = 11
>>> release = date(2023, 10, 2)
>>> f"Python {major}.{minor + 1} is released on {release:%B %-d}"
'Python 3.12 is released on October 2'

This f-string contains three pairs of curly braces ({}). Each pair contains an expression that’s interpolated into the final string. The first expression only refers to major directly, while the second one applies a small operation on minor.

The third expression shows that you can add certain format specifiers to control the interpolation of an expression. In this case release is a date, so %B and %-d are interpreted as date format specifiers, formatting the date as month day.

F-strings were originally implemented with a dedicated parser. That means that even though expressions inside f-strings are regular Python expressions, the expressions weren’t parsed with the regular Python parser. Instead, the developers implemented a separate parser, which they needed to maintain.

The primary reason for this was the inabilty of Python’s old LL(1) parser to support f-strings. After the introduction of the PEG parser in Python 3.9, this is no longer true. The current parser could parse expressions within f-strings.

Based on PEP 701, f-strings are now formalized as additions to Python’s grammar in Python 3.12. In practice, this means that the PEG parser will parse f-strings, just like regular Python code.

For the most part, you won’t notice this change to f-strings. It mainly benefits the core developers maintaining Python’s source code. However, there are a few changes visible to anyone using f-strings.

In general, the new implementation of f-strings lifts some restrictions that were added originally. Many of these restrictions were put in place to make f-strings easier to handle for external tools like IDEs and code highlighters. Below, you’ll explore some examples that weren’t possible before.

You may now reuse the string quote character inside of the f-string. For example, if you’ve delimited your f-string with double-quotes ("), you can still use " inside expressions:

Python
>>> version = {"major": 3, "minor": 12}
>>> f"Python {version["major"]}.{version["minor"]}"
'Python 3.12'

Even though double-quotes delimit the f-string, you can still use " to specify the keys inside the f-string.

Until now, you haven’t been able to use a backslash character (\) inside an f-string expression. Going forward, you can use backslashes in f-string expressions as in any other expression:

Python
>>> names = ["Brett", "Emily", "Gregory", "Pablo", "Thomas"]
>>> print(f"Steering Council:\n {"\n ".join(names)}")
Steering Council:
 Brett
 Emily
 Gregory
 Pablo
 Thomas

Here, you use \n, which represents a newline character, in both the string and expression parts of the f-string. Previously, the latter wasn’t allowed.

As with other types of braces and parentheses, you can now add newlines inside the curly braces delimiting expressions in f-strings. As an added bonus, you can also add comments to expressions. Since a comment extends to the end of the line, you need to close the expression on the next line or later.

To see how this works, continue the example from above:

Python
>>> f"Steering council: {
...   ", ".join(names)  # Steering council members
... }"
'Steering council: Brett, Emily, Gregory, Pablo, Thomas'

The highlighted line shows a commented f-string expression on a line by itself.

While the important changes to f-strings have happened mostly under the hood, you’ve seen that some dusty corners have been improved, and f-string expressions are now more consistent with the rest of Python.

If you want to dive deeper into the changes in f-strings, have a look at Python 3.12 Preview: More Intuitive and Consistent F-Strings.

Faster Python: More Specializations and Inlined Comprehensions

When Python 3.11 came out in 2022, there was a lot of buzz about optimizations to the interpreter that made Python faster. That work was part of an ongoing effort named faster-cpython, and it’s continued into Python 3.12.

Before a Python script starts running, the code is translated into bytecode. The bytecode is the code that the Python interpreter runs. Python 3.11 uses a specialized adaptive interpreter that can change and adapt the bytecode during execution in order to optimize operations that happen often. This depends on two steps:

  • Quickening is the process of noticing that certain bytecode is executed several times, making it a candidate for specialization.
  • Specialization means that the interpreter replaces a general bytecode with a specialized one. For example, an operation that adds two floating-point numbers can replace a general addition operation.

In Python 3.12, quickening happens faster than in Python 3.11, and the interpreter can now specialize many new bytecodes. To see the quickening and specialization in action, define the following function:

Python
>>> def feet_to_meters(feet):
...     return 0.3048 * feet
...

You can use feet_to_meters() to convert from feet to meters. To look behind the curtain of the interpreter, you’ll use dis, which lets you disassemble your Python code and look at the bytecode instead:

Python
>>> import dis
>>> dis.dis(feet_to_meters, adaptive=True)
  1           0 RESUME                   0

  2           2 LOAD_CONST__LOAD_FAST    1 (0.3048)
              4 LOAD_FAST                0 (feet)
              6 BINARY_OP                5 (*)
             10 RETURN_VALUE

Each line shows information about one bytecode instruction. The five columns are the line number, the byte address, the operation code name, the operation parameters, and an interpretation of the parameters in parentheses.

You don’t need to understand the details of this bytecode listing. Still, note that one line of Python code often compiles to several bytecode instructions. In this example, return 0.3048 * feet translates to four bytecode instructions.

Actually, there’s no separate quickening step any longer. In principle, all bytecode instructions are immediately ready for specialization. In Python 3.11, the specialization kicked in after a bytecode had executed with the same types eight times. Now, this happens already after two calls:

Python
>>> feet_to_meters(1.1)
0.33528
>>> feet_to_meters(2.2)
0.67056

You call feet_to_meters() twice, each time with a float argument. The interpreter will then specialize and assume that the multiplication will continue to be between floating point numbers:

Python
>>> dis.dis(feet_to_meters, adaptive=True)
  1           0 RESUME                    0

  2           2 LOAD_CONST__LOAD_FAST     1 (0.3048)
              4 LOAD_FAST                 0 (feet)
              6 BINARY_OP_MULTIPLY_FLOAT  5 (*)
             10 RETURN_VALUE

The interpreter has adapted the original BINARY_OP instruction, replacing it with BINARY_OP_MULTIPLY_FLOAT, which is faster when both operands are float.

Even though the interpreter is adapting certain bytecodes, this doesn’t harm Python’s dynamic nature. You can still use feet_to_meters() with integer arguments. The only bonus to using the same data types is that your program may run faster.

To learn more, check out core developer Brandt Bucher’s presentation at PyCon 2023: Inside Python’s new specializing, adaptive interpreter.

PEP 709 describes a new optimization in Python 3.12: inlined comprehensions. Python supports list comprehensions, dictionary comprehensions, and set comprehensions that you use to transform iterables. For example:

Python
>>> names = ["Brett", "Emily", "Gregory", "Pablo", "Thomas"]
>>> [name[::-1].title() for name in names]
['Tterb', 'Ylime', 'Yrogerg', 'Olbap', 'Samoht']

Here, you use a list comprehension to reverse each of the names. Such a comprehension is currently compiled as a nested function. To explore this, first wrap the comprehension in a function:

Python
>>> def reverse_names(names):
...     return [name[::-1].title() for name in names]
...

Similarly to the example above, you reverse each name and make sure that it starts with a capital letter. Now, use dis to disassemble the function on Python 3.11:

Python
>>> import dis
>>> dis.dis(reverse_names)
  1       0 RESUME              0

  2       2 LOAD_CONST          1 (<code object <listcomp> at 0x7f2e61f42d30)
          4 MAKE_FUNCTION       0
          6 LOAD_FAST           0 (names)
          8 GET_ITER
         10 PRECALL             0
         14 CALL                0
         24 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x7f2e61f42d30>:
  2       0 RESUME              0
          2 BUILD_LIST          0
          4 LOAD_FAST           0 (.0)
    >>    6 FOR_ITER           31 (to 70)
          8 STORE_FAST          1 (name)
         10 LOAD_FAST           1 (name)
         12 LOAD_CONST          0 (None)
         14 LOAD_CONST          0 (None)
         16 LOAD_CONST          1 (-1)
         18 BUILD_SLICE         3
         20 BINARY_SUBSCR
         30 LOAD_METHOD         0 (title)
         52 PRECALL             0
         56 CALL                0
         66 LIST_APPEND         2
         68 JUMP_BACKWARD      32 (to 6)
    >>   70 RETURN_VALUE

There are many details in this listing that you can ignore. The important thing to notice is that a new listcomp code object has been created. In the top part of the bytecode listing, you can see that this internal function has to be loaded and then called.

Compiling a comprehension into a nested function like this is convenient, as the function call isolates the comprehension such that it doesn’t leak variables. However, it’s not necessarily the most efficient implementation. Especially if the comprehension runs over a small iterable, then the overhead of calling the nested function is noticeable.

In Python 3.12, comprehensions are inlined into the bytecode. Have a look at the disassembly of reverse_names() in the new version:

Python
>>> dis.dis(reverse_names)
  1           0 RESUME                   0

  2           2 LOAD_FAST                0 (names)
              4 GET_ITER
              6 LOAD_FAST_AND_CLEAR      1 (name)
              8 SWAP                     2
             10 BUILD_LIST               0
             12 SWAP                     2
        >>   14 FOR_ITER                24 (to 66)
             18 STORE_FAST               1 (name)
             20 LOAD_FAST                1 (name)
             22 LOAD_CONST               0 (None)
             24 LOAD_CONST               0 (None)
             26 LOAD_CONST               1 (-1)
             28 BUILD_SLICE              3
             30 BINARY_SUBSCR
             34 LOAD_ATTR                1 (NULL|self + title)
             54 CALL                     0
             62 LIST_APPEND              2
             64 JUMP_BACKWARD           26 (to 14)
        >>   66 END_FOR
             68 SWAP                     2
             70 STORE_FAST               1 (name)
             72 RETURN_VALUE

Again, the details of the bytecode aren’t important. Instead, note that there’s no extra code object, and no extra function call happens.

In general, the inlined comprehensions are faster than the previous implementation. Comprehensions running over small iterables can be as much as twice as fast as before. If the comprehension runs over a larger iterable with thousands of elements, then you may notice that the new implementation is on par with or even slightly slower than in Python 3.11. However, real-world benchmarks suggest that you may expect a speedup in your code.

You can find code that benchmarks different comprehensions in the accompanying materials that you can download for this tutorial. Use this to check the performance of comprehensions on your computer:

The effort to make Python faster continues, and there are already many ideas slated for Python 3.13.

Dedicated Type Variable Syntax

Python added support for annotations in version 3.0. While type hinting was one of the motivations for annotations, Python’s support for static typing wasn’t in place until Python 3.5, several years later.

Type variables constitute an important and powerful part of Python’s typing system. A type variable can stand in for a concrete type during static type checking. You use type variables to parametrize generic classes and generic functions. Consider the following example, which returns the first element in a given list:

Python
def first(elements):
    return elements[0]

The type of the return value of first() depends on the kind of list that you pass in. For example, if elements is a list of integers, then first() returns int, while if it’s a list of strings, then the return type is str. You’ll use type variables to express this relationship.

Python 3.12 introduces a new syntax for type variables. With the new syntax, you can write the following:

Python
def first[T](elements: list[T]) -> T:
    return elements[0]

By adding T inside square brackets to the function definition, you declare that first() is a generic function parametrized by the type variable T.

Declaring first() as a generic function has no effect at runtime. Instead, it helps your editor or static type checker in tracking the types that you use. Look at the following two examples:

Python
>>> def first[T](elements: list[T]) -> T:
...     return elements[0]
...

>>> first([2, 3, 1, 0])
2

>>> first(["Brett", "Emily", "Gregory", "Pablo", "Thomas"])
'Brett'

In the first invocation of first(), you pass in a list of integers. In this case, a type checker will see that T can be int and will deduce that first() returns an int. In the second example, you pass in a list of string names. In this case, elements is list[str], so T will be treated as str.

As noted, type variables have been available for a long time. What Python 3.12 brings to the table is the new and powerful syntax for using them. Previously, you’d implement the same example by importing TypeVar from typing:

Python
from typing import TypeVar

T = TypeVar("T")

def first(elements: list[T]) -> T:
    return elements[0]

There are two main advantages to the new syntax. First of all, it’s part of Python’s regular grammar, so you don’t need to import the TypeVar class. Additionally, T is declared in the function definition, instead of outside the function. This makes the role of the type variable more clear.

You’ve seen the most straightforward use of type variables. The new syntax supports other uses as well. These include multiple type variables, constrained type variables, and bounded type variables, in addition to generic classes and generic type aliases. You’ll explore these use cases in the following examples.

You often use tuples to represent a heterogeneous sequence with a predetermined number of elements. A basic example would be a tuple representing information about a person as a name-age pair. In terms of types, you could describe these tuples as tuple[str, int].

Now, say that you have a function that flips the order of such tuple pairs. In general, the types of the two elements will be different, so you’d need two type variables to represent them. You can declare two or more type variables inside the square brackets, separated by commas:

Python
def flip[T0, T1](pair: tuple[T0, T1]) -> tuple[T1, T0]:
    first, second = pair
    return (second, first)

Here, T0 and T1 are two independent type variables. They can take on different types, but they may also be the same. For example, maybe you pass in a pair of Booleans.

By default, type variables can be materialized by any type. However, sometimes you want to express type relationships that are constrained to one of just a few types or bounded as a subtype of some type. You can do so by adding a condition after the type variable, separated by a colon. You’ll use the following syntax:

Python
>>> def free[T](argument: T) -> T: ...

>>> def constrained[T: (int, float, complex)](argument: T) -> T: ...

>>> def bounded[T: str](argument: T) -> T: ...

In the examples so far, you’ve used the free syntax. This implies that T can be any type. In constrained(), T is a type variable that can only be int, float, or complex. You express this by using a literal tuple of types. In bounded(), T can be str or any subclass of str.

You can also define generic classes. As for generic functions, you declare any type variables in square brackets. The following example implements a simple stack based on list:

Python
# generic_stack.py

class Stack[T]:
    def __init__(self) -> None:
        self.stack: list[T] = []

    def push(self, element: T) -> None:
        self.stack.append(element)

    def pop(self) -> T:
        return self.stack.pop()

Here you’ve added [T] to the class name. Then, you can use the type variable when you’re annotating the method parameters and return types inside the class. In practice, you can instantiate stacks containing specific types. Next, you’ll create a stack of integers:

Python
>>> from generic_stack import Stack
>>> numbers = Stack[int]()
>>> numbers.push(3)
>>> numbers.push(12)

>>> numbers.stack
[3, 12]

>>> numbers.pop()
12

By adding [int] when you instantiate Stack, you’re telling the type checker that your data structure will be composed of integers. It can then warn you if other types may end up inside your stack.

Observe that when you pop a number off your stack, you get the last number that was pushed. This is often referred to as last-in, first-out (LIFO). The idea is reminiscent of a stack of plates that you may have in your kitchen cupboard. Stacks are useful in many different computer algorithms as well.

You can also use a new syntax for type aliases. A type alias is a name that you give to a specific type, either to simplify working with a nested data type, or to more explicitly describe your data type.

You can now use type to declare type aliases:

Python
>>> type Person = tuple[str, int]
>>> type ListOrSet[T] = list[T] | set[T]

Here Person is a type represented by a tuple consisting of a string and an integer. ListOrSet is a generic type alias, which a list or a set will represent. You can later annotate an argument with something like ListOrSet[int] which would require the argument to be either a list of integers or a set of integers.

You can learn more about the new syntax for type variables and see more practical examples in Python 3.12 Preview: Static Typing Improvements and PEP 695.

Support for the Linux perf Profiler

A profiler is a tool for monitoring and diagnosing the performance of your scripts and programs. By profiling your code, you’ll get accurate measurements that you can use to tune your implementation.

Python has long supported profiling with tools like timeit and cProfile in the standard library. Additionally, third-party tools like line-profiler, Pyinstrument, and Fil provide other capabilities.

The perf profiler is a profiler built into the Linux kernel. While it’s only available on Linux, it’s a popular and powerful tool that can provide information about everything from hardware events and system calls to running library code.

Until now, running perf hasn’t worked well with Python. The CPython interpreter is the program that usually runs your Python code. Python code is evaluated with a C function named _PyEval_EvalFrameDefault(), and a typical profile of a Python program will only show that it spent most of the time in _PyEval_EvalFrameDefault().

Python 3.12 adds proper support for perf and gives it the ability to monitor Python functions through a technique called trampoline instrumentation. This allows individual Python functions to show up in profiling reports that perf produces:

Text
Samples: 10K of event 'cycles', Event count (approx.): 34217160648

-  100.00%     10432
   -   99.16%     /home/realpython/python-custom-build/bin/python3.12
        __libc_start_main
      - Py_BytesMain
         - 99.89% pymain_run_python.constprop.0
              _PyRun_AnyFileObject
              _PyRun_SimpleFileObject
              run_mod
              run_eval_code_obj
              PyEval_EvalCode
              py::<module>:/home/realpython/project/script.py
              _PyEval_EvalFrameDefault
              PyObject_Vectorcall
              py::main:/home/realpython/project/script.py
              _PyEval_EvalFrameDefault
            - PyObject_Vectorcall
               + 66.61% py::slow_function:/home/realpython/project/script.py
               + 33.39% py::fast_function:/home/realpython/project/script.py
   +    0.84%     /proc/kcore

If you’re running Linux and are interested in profiling your code, then you should give perf a try. For more information, including how to set up perf on your system and profile your code, check out Python 3.12 Preview: Support for the Linux Perf Profiler.

Other Pretty Cool Features

Until now, you’ve seen the biggest changes and improvements in Python 3.12. However, there’s much more to explore. In this section, you’ll take a look at some of the new features that may be sneaking under the headlines. They include more internal changes to the interpreter, a new typing feature, and new functions for grouping iterables and listing files.

One GIL Per Subinterpreter

Python has a global interpreter lock (GIL) that simplifies a lot of internal code in the interpreter. At the same time, the GIL imposes some restrictions on running concurrent Python code. In particular, only one thread is usually allowed to run at a time, which makes parallel processing cumbersome.

Over time, there’ve been several initiatives to remove the GIL from the language. Recently, PEP 703 and the nogil project have caused a lot of buzz, and there’s a roadmap for removing the GIL from Python.

A related initiative is seeing the light in Python 3.12. PEP 684 describes a per-interpreter GIL. The Python interpreter is the program that executes your Python scripts and programs. It’s possible to spawn new interpreters, so-called subinterpreters, but you can only do so in extension modules through the C API.

Having a per-interpreter GIL means that there’s a separate interpreter lock for each subinterpreter. This opens up the possibility of new and efficient ways of doing parallelism in Python that take better advantage of the multiple cores found in modern computers. One such interesting model is communicating sequential processes (CSP) which has inspired concurrency in several languages, including Erlang and Go.

To achieve a per-interpreter GIL, the core developers have refactored several parts of the CPython internals. Python has both global state storage and per-interpreter storage. In this initiative, much of what was previously stored as global state is now stored for each interpreter.

Probably, you won’t notice this change when running Python 3.12. None of the changes are exposed to regular users of Python. Instead, they’re laying the groundwork for new ways to implement parallelism in the future.

You can learn more about subinterpreters, including the plans for making them more accessible in Python 3.13, in Python 3.12 Preview: Subinterpreters.

Immortal Objects

The introduction of immortal objects into Python is another internal feature that improves the CPython interpreter and prepares the way for new developments in the future.

For efficiency, several objects in Python are singletons. For example, there’s only one None object during program execution, independent of how many times you refer to None in your code. This optimization is possible because None is immutable. It’ll never change its value.

It turns out that while None is immutable as seen from Python’s perspective, the None object handled by the CPython interpreter does change. The interpreter represents every object in a struct that includes the object’s reference count in addition to the object’s data. The reference count changes every time your code references an object.

You can check an object’s reference count for yourself:

Python
>>> import sys
>>> a = 3.12
>>> sys.getrefcount(a)
2

>>> b = a
>>> sys.getrefcount(a)
3

>>> del b
>>> sys.getrefcount(a)
2

You use sys.getrefcount() to inspect the reference count of an object. Here a refers to the float object 3.12. You can see that the reference count increases when b refers to the same object. Likewise, the reference count decreases when the name b is deleted.

The reference count is important for Python’s memory management. CPython uses a garbage collector that removes objects from memory once their reference count hits zero.

Immortal objects are objects which are truly immutable, including inside the interpreter. This means that their reference count doesn’t change. Immortal objects are identified by having their reference count set to a special flag. This is done to keep backwards compatibility of the C-API and to mostly treat immortal objects the same as regular objects.

You can see this mechanism if you look at the reference count of an immortal object:

Python
>>> import sys
>>> sys.getrefcount(None)
4294967295

>>> a = None
>>> sys.getrefcount(None)
4294967295

At first, you get the impression that None is referenced more than four billion times. However, 4,294,967,295 is a special value indicating that None is an immortal object. Note that the number doesn’t change when you create a new reference to None.

The special flag isn’t chosen at random. It corresponds to the hexadecimal number FFFFFFFF, which is the largest integer that a 32-bit system can natively represent:

Python
>>> hex(sys.getrefcount(None))
'0xffffffff'

In other words, the immortal flag value is big enough so that the interpreter won’t reach it as a reference count in normal use. And it has a representation that’s efficient to test against.

While immortal objects are an implementation detail in the interpreter, they bring a couple of advantages:

  • Truly immutable objects have better cache behavior. In certain code, it’ll be more efficient to work with singletons like None, True, False, and so on.
  • Immortal objects don’t need the GIL’s protection. Therefore, they’ll work well with the per-interpreter GIL that you learned about earlier. They also simplify some of the work towards removing the GIL from Python.

You can learn more about immortal objects in PEP 683. Furthermore, Introducing Immortal Objects for Python provides some context and background for the implementation, while Understanding Immortal Objects in Python 3.12 discusses the implementation itself.

Override Decorator for Explicit Inheritance

Python is an object-oriented language with good support for working with classes and inheritance. There’s a close connection between Python’s classes and its static typing system, as each class also defines a type that you can use in type hints.

One of the new typing features in Python 3.12 is @override. You can use this decorator to mark methods in subclasses that override methods in the parent class. This feature is partly inspired by similar mechanisms in other languages like Java and C++.

Using @override can help you avoid certain kinds of bugs, especially when you refactor your code. A static type checker can warn you in the following cases:

  • When you rename a method but forget to rename the corresponding methods in subclasses
  • When you misspell the name of a subclass method that’s supposed to override a parent method
  • When you add a new method to a class that accidentally overrides an existing method in a subclass

Until now, type checkers had no way of knowing whether a method was meant to override another or not. In Python 3.12, @override is added to typing. On earlier versions of Python, you can import @override from the third-party typing-extensions library.

For an example of how you can use the new decorator, you’ll implement two classes representing bank accounts. BankAccount will represent a basic bank account, while SavingsAccount will subclass BankAccount with some special features.

For simplicity, you’ll use data classes to define your bank account classes. Start with the general bank account class:

Python
# accounts.py

import random
from dataclasses import dataclass
from typing import Self

def generate_account_number() -> str:
    """Generate a random eleven-digit account number"""
    account_number = str(random.randrange(10_000_000_000, 100_000_000_000))
    return f"{account_number[:4]}.{account_number[4:6]}.{account_number[6:]}"

@dataclass
class BankAccount:
    account_number: str
    balance: float

    @classmethod
    def from_balance(cls, balance: float) -> Self:
        return cls(generate_account_number(), balance)

    def deposit(self, amount: float) -> None:
        self.balance += amount

    def withdraw(self, amount: float) -> None:
        self.balance -= amount

Your bank account has two regular metods: .deposit() and .withdraw(). Additionally, you add an alternative constructor, .from_balance(), that can create a new bank account given an initial balance. When you do this, the generate_account_number() utility function generates a random account number.

You can use this bank account as follows:

Python
>>> from accounts import BankAccount
>>> account = BankAccount.from_balance(1000)
>>> account.withdraw(123.45)
>>> account
BankAccount(account_number='2801.83.60525', balance=876.55)

Here, you start a new bank account with an initial balance of $1000. After withdrawing $123.45, you have $876.55 left in the account.

Next, you’ll add a savings account. Compared to the regular bank account, the savings account can accrue interest. Additionally, the bank provide a small bonus to the account holder by rounding withdrawals bigger than $100 down to the nearest dollar amount.

You implement SavingsAccount as a subclass of BankAccount and mark the methods that override the parent methods. Make sure to add override to your imports:

Python
# accounts.py

import random
from dataclasses import dataclass
from typing import Self, override

# ...

@dataclass
class SavingsAccount(BankAccount):
    interest: float

    def add_interest(self) -> None:
        self.balance *= (1 + self.interest / 100)

    @classmethod
    @override
    def from_balance(cls, balance: float, interest: float = 1.0) -> Self:
        return cls(generate_account_number(), balance, interest)

    @override
    def withdraw(self, amount: float) -> None:
        self.balance -= int(amount) if amount > 100 else amount

First, you add .interest as a new attribute and .add_interest() as a new method for adding interest to the savings account. Next, you update .from_balance() to support specifying the interest. Since this constructor overrides the corresponding method in BankAccount, you mark it with @override.

You also override .withdraw() to add a small bonus for account holders. If a customer withdraws more than $100, then you’ll only subtract the amount rounded down to the nearest dollar from the balance:

Python
>>> from accounts import SavingsAccount
>>> savings = SavingsAccount.from_balance(1000, interest=2)
>>> savings.withdraw(123.45)
>>> savings
SavingsAccount(account_number='5283.78.04835', balance=877, interest=2)

Here, you’ve only subtracted $123 from the balance even though you withdrew $123.45. This shows that SavingsAccount uses the overridden method. You can continue the example to test the new method:

Python
>>> savings.add_interest()
>>> savings
SavingsAccount(account_number='5283.78.04835', balance=894.54, interest=2)

>>> savings.withdraw.__override__
True

After adding interest to the account, you also confirm that you’ve tagged .withdraw() as an override. Even though you’ve added the .__override__ attribute, this won’t have any effect. Instead, you should use a static type checker to help you catch errors in overriding methods. Have a look at Python 3.12: Static Typing Improvements for more details on how you can activate your type checker.

If you’re interested in catching similar issues at runtime, then check out the third-party library overrides.

Calendar Constants for Days and Months

Python’s calendar module is one of the batteries that’s been included in the language for a long time. Usually, when you’re working with dates, you’ll use the datetime module, which provides date and datetime classes that represent dates and dates with timestamps, respectively.

With calendar, you can bridge the gap between the technical use of datetime and the often more user-friendly representations of a calendar. For example, you can use calendar to quickly show a calendar in your terminal:

Shell
$ python -m calendar 2023 10
    October 2023
Mo Tu We Th Fr Sa Su
                   1
 2  3  4  5  6  7  8
 9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

By running the module with -m, you display a calendar. In the example above, you show October 2023 and observe that October 2 is a Monday.

You can also use calendar in your own code if you need to build custom calendars. In the new Python version, you have more support for working with weekdays and months. First, look at what’s been available for a long time, like lists of weekday and month names:

Python
>>> import calendar

>>> ", ".join(calendar.month_abbr)
', Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec'

>>> calendar.weekday(2023, 10, 2)
0
>>> calendar.day_name[0]
'Monday'

You first list all months, abbreviated. Note that the list of months has thirteen elements, where the first one is an empty string. This is done so that indices match the usual month numbers, where January corresponds to 1 and so on.

In the second example, you use calendar.weekday() to calculate which weekday October 2 falls on in 2023. You then look up that answer—0—in the list of day names to confirm that October 2 is a Monday.

Everything that you’ve seen so far is possible on older versions of Python. In Python 3.12, you’ll find two minor additions, namely the enumerations Day and Month. An enumeration is a collection of constant values in a common namespace. These make it more convenient to handle weekdays and months.

Enumeration members are constants. You can access them in a few different ways:

Python
>>> calendar.Month(10)
calendar.OCTOBER

>>> calendar.Month.OCTOBER
calendar.OCTOBER

>>> calendar.Month["OCTOBER"]
calendar.OCTOBER

First, you ask for the tenth month by calling the enumeration. Then, you ask specifically for the .OCTOBER constant by using dot notation. Finally, you look up "OCTOBER" as if it were a key on Month.

All three of these return calendar.OCTOBER, which is a constant with the value 10. You can use the enumeration and its value interchangably:

Python
>>> october = calendar.Month(10)
>>> october
calendar.OCTOBER

>>> october.name
'OCTOBER'

>>> october.value
10

>>> october + 1
11

>>> calendar.month_abbr[october]
'Oct'

You can explicitly look up the name and value of the enumeration by using .name and .value, respectively. In addition, you can make calculations or use the enumeration in expressions as if it were a regular value.

Day works similarly to Month, but it contains weekday constants. Monday is represented by 0, Tuesday by 1, and so on up to Sunday, which corresponds to 6. In Python 3.12, the weekday() function returns a Day enumeration:

Python
>>> release_day = calendar.weekday(2023, 10, 2)
>>> release_day
calendar.MONDAY

>>> release_day.name
'MONDAY'

>>> release_day.value
0

Here, you represent release_day with a Day enumeration. As above, you can peek at its name and value.

One final feature of enumerations is that you can iterate over them. In the next example, you loop over Month to create a quick sales report:

Python
>>> sales = {1: 5, 2: 9, 3: 6, 4: 14, 5: 9, 6: 8, 7: 15, 8: 22, 9: 20, 10: 23}
>>> for month in calendar.Month:
...     if month in sales:
...         print(f"{month.value:2d} {month.name:<10} {sales[month]:2d}")
...
 1 JANUARY     5
 2 FEBRUARY    9
 3 MARCH       6
 4 APRIL      14
 5 MAY         9
 6 JUNE        8
 7 JULY       15
 8 AUGUST     22
 9 SEPTEMBER  20
10 OCTOBER    23

Each month is an enumeration constant. Note how you can treat the constant like a regular integer when you check if month is a key in sales and when you use month as an index to sales.

Similarly to months, you can iterate over weekdays:

Python
>>> ", ".join(day.name for day in calendar.Day)
'MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY'

In this example, you combine the names of all weekdays into a string. However, if you need to work with the weekday names, using calendar.day_name or calendar.day_abbr might still be the better option. These give you localized names, which means that they’ll be translated into your local language.

itertools.batched(): Group Items in an Iterable

The itertools standard library module includes many powerful functions for working with and manipulating iterables. For example, you can calculate all the combinations of two weekdays:

Python
>>> import itertools
>>> list(itertools.combinations(["Mo", "Tu", "We", "Th", "Fr", "Sa", "Su"], 2))
[('Mo', 'Tu'), ('Mo', 'We'), ('Mo', 'Th'), ('Mo', 'Fr'), ('Mo', 'Sa'),
 ('Mo', 'Su'), ('Tu', 'We'), ('Tu', 'Th'), ('Tu', 'Fr'), ('Tu', 'Sa'),
 ('Tu', 'Su'), ('We', 'Th'), ('We', 'Fr'), ('We', 'Sa'), ('We', 'Su'),
 ('Th', 'Fr'), ('Th', 'Sa'), ('Th', 'Su'), ('Fr', 'Sa'), ('Fr', 'Su'),
 ('Sa', 'Su')]

This lists the twenty-one ways to pair up two weekdays. There are many other functions available for you to play with to work efficiently with iterables.

A frequently requested function has been batched(), which can group the elements of an iterator into batches of a given size. Until now, you could implement batched() yourself based on the official itertools recipes, or you could rely on the third-party more-itertools library.

In Python 3.12 batched() has been included in itertools, making it more readily available. The function returns an iterator that yields tuples of the given length. The final batch may be smaller than the others. In the following example, you group the first ten integers into batches of four numbers:

Python
>>> import itertools
>>> for batch in itertools.batched(range(10), n=4):
...     print(batch)
...
(0, 1, 2, 3)
(4, 5, 6, 7)
(8, 9)

The first two batches consist of four numbers each. However, the last batch contains only the two numbers left over in the range.

In many cases, you’ll work with iterables that you know divide cleanly into batches. Picking up from the previous section, you may want to group the months into quarters. You can use batched() as follows:

Python
>>> import calendar
>>> import itertools
>>> for quarter in itertools.batched(calendar.Month, n=3):
...     print([month.name for month in quarter])
...
['JANUARY', 'FEBRUARY', 'MARCH']
['APRIL', 'MAY', 'JUNE']
['JULY', 'AUGUST', 'SEPTEMBER']
['OCTOBER', 'NOVEMBER', 'DECEMBER']

You iterate over the new Month enumeration and group the months into batches of three months. You use a list comprehension to print out the name of each month. Each list in your output represents the months in a quarter.

The new batched() function adds another powerful tool to the itertools library. If you work with iterables, you should make sure that you’re familiar with the possibilities in itertools.

Path.walk(): List Files and Subdirectories

Another useful module in the standard library is pathlib. You use pathlib to work with paths in your file system, and you can use it read and write files efficiently:

Python
>>> from pathlib import Path
>>> names = ["Brett", "Emily", "Gregory", "Pablo", "Thomas"]

>>> council_path = Path("council.txt")
>>> council_path.write_text("\n".join(names), encoding="utf-8")
32

Here, you create a path to a file named council.txt in your working directory. Then you write a list of names separated by newlines to that file.

While a Path object has many methods for manipulating paths and creating new paths, it only has limited support for listing files and directories. You can use .glob(), but this method really shines when you’re looking for files and directories with names matching a specific pattern:

Python
>>> for path in Path.cwd().glob("*.txt"):
...     print(path)
...
/home/realpython/council.txt

You use .glob() to list all the files ending with the .txt suffix in the current working directory (cwd). You can use the recursive counterpart, .rglob(), to list and filter all the files in subdirectories as well.

In Python 3.12, you can use the new .walk() method to work with files and directories. Assume that you have some information about musicians stored in the following file hierarchy:

musicians/
│
├── trumpet/
│   ├── armstrong.txt
│   └── davis.txt
│
├── vocal/
│   └── fitzgerald.txt
│
└── readme.txt

First, use .rglob() to list all files and directories recursively:

Python
>>> for path in sorted(Path.cwd().rglob("*")):
...     print(path)
...
/home/realpython/musicians
/home/realpython/musicians/readme.txt
/home/realpython/musicians/trumpet
/home/realpython/musicians/trumpet/armstrong.txt
/home/realpython/musicians/trumpet/davis.txt
/home/realpython/musicians/vocal
/home/realpython/musicians/vocal/fitzgerald.txt

This gives you one path for each file or directory in your hierarchy. In general, the order of paths isn’t deterministic when you use .glob() and .rglob(). One way to keep your file listings reproducible is to sort them with sorted().

The new .walk() method works slightly differently, as it focuses on directories:

Python
>>> for path, directories, files in Path.cwd().walk():
...     print(path, directories, files)
...
/home/realpython ['musicians'] []
/home/realpython/musicians ['trumpet', 'vocal'] ['readme.txt']
/home/realpython/musicians/trumpet [] ['armstrong.txt', 'davis.txt']
/home/realpython/musicians/vocal [] ['fitzgerald.txt']

Note that .walk() yields tuples of three elements. The path will always refer to a directory. The last two elements are lists of subdirectories and files directly inside that directory, respectively. You can use the top_down parameter to control the order in which directories are listed.

The new method is based on os.walk(). The main difference is that the new .walk() yields Path objects.

So, Should You Upgrade to Python 3.12?

You’ve seen the coolest new features and improvements in Python 3.12. The next question might be whether you should upgrade to the new version, and if so, when should you upgrade?

As always, the not-so-helpful answer is it depends!

A small start is to install Python 3.12 alongside your current system. This allows you to start playing with the new features when doing local development. This comes with minimal risk because any bugs that you encounter should have limited effect. At the same time, you get to go ahead and take advantage of the improved error messages and optimizations to the interpreter.

You should be more careful with updating any production environment that you control, as the consequences of bugs and errors are more severe in that context. All new Python releases are well tested in the beta phase. Still, it might be a good idea to wait for the first few bugfix releases before switching.

One possible problem with updating to Python 3.12 is that you depend on a third-party library that’s not ready for the new version. In particular, packages that use C-extensions must be compiled specially for version 3.12, and this may take some time. Luckily, this is less of a problem than it used to be, as more and more package maintainers update their packages ahead of the release.

So far, you’ve considered when you can start using the new interpreter. Another important question is when you can start to take advantage of the updated syntax and new features in the language. If you’re maintaining a library that needs to support older versions of Python, then you can’t use the new type variable syntax or the improvements to f-strings. You need to stick with code compatible with the older versions.

The situation is different if you’re developing an application where you control the environment that it’s running in. In that case, you can upgrade the environment to Python 3.12 as soon as your dependencies are available, and then start using the new syntax.

Conclusion

A new version of Python is always a great occasion to celebrate your favorite language and all the volunteers that put time and effort into its development. Thanks to the work of so many developers, Python 3.12 brings several improvements to the table.

In this tutorial, you’ve seen new features and improvements, like:

  • Better error messages with helpful suggestions and guidance
  • More expressive f-strings that are backed by Python’s PEG parser
  • Optimizations, including inlined comprehensions, to help Python run faster
  • A new syntax for type variables that you use to annotate generics
  • Support for the powerful perf profiler on Linux

While you may not be able to take advantage of all these features right away, you can install Python 3.12 and play with them. If you’re thirsty for more information about the new release, then check out the comprehensive podcast episode and these tutorials, which focus on one new feature each:

It’s also a good time to start testing your existing code on Python 3.12 to make sure that it’s ready for the future.

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: What's New in Python 3.12

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Geir Arne Hjelle

Geir Arne Hjelle Geir Arne Hjelle

Geir Arne is an avid Pythonista and a member of the Real Python tutorial team.

» More about Geir Arne

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!

Keep Learning

Related Tutorial Categories: intermediate python

Recommended Video Course: What's New in Python 3.12