In February 2022, another preview release of Python 3.11 became available for early adopters to test and share their feedback. Shortly afterward, the Python steering council announced that Python 3.11 would also include a TOML parser in the standard library. In addition, GitHub Issues will soon become the official bug tracking system for Python.
In other news, PyCon US 2022 shared its conference schedule. The Python Software Foundation (PSF) wants to hire two contract developers to improve the Python Package Index (PyPI). Apple is finally removing Python 2.7 from macOS.
Let’s dive into the biggest Python news from the past month!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
Python 3.11 Alpha 5 Released
While the final release of Python 3.11 is planned for October 2022, which is still months ahead, you can already check out what’s coming. According to the development and release schedule described in PEP 664, Python 3.11 is now in the alpha phase of its release cycle, which is meant to collect early feedback from users like you. At the beginning of February, 3.11.0a5, the fifth of seven planned alpha releases, became available for testing.
To play around with an alpha release of Python, you can grab the corresponding Docker image from Docker Hub, install an alternative Python interpreter with pyenv, or build the CPython source code using a compiler tool. The source code method lets you clone CPython’s repository from GitHub and check out a bleeding-edge snapshot without waiting for an alpha release.
Note: Features under development might be unstable and buggy, and they can be modified or removed from the final release without notice. As a result of that, you should never use preview releases in a production environment!
If you’d like to explore some of the most exciting features coming to Python 3.11, then make sure you can run the alpha release of the interpreter. You’ll find the commands to download and run Python 3.11.0a5 below:
$ docker pull python:3.11.0a5-slim
$ docker run -it --rm python:3.11.0a5-slim
Python 3.11.0a5 (main, Feb 25 2022, 20:02:52) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ pyenv update
$ pyenv install 3.11.0a5
$ pyenv local 3.11.0a5
$ python
Python 3.11.0a5 (main, Mar 1 2022, 10:05:02) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ git clone git@github.com:python/cpython.git
$ cd cpython/
$ git checkout v3.11.0a5
$ ./configure
$ make
$ ./python
Python 3.11.0a5 (tags/v3.11.0a5:c4e4b91557, Mar 1 2022, 17:48:56) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
So what’s all the hype about this alpha release? There are several big and small improvements, but for now, let’s focus on some of the Python 3.11 highlights!
PEP 657: Error Locations in Tracebacks
Python 3.10 has already greatly improved its error messages. By pinpointing the root cause, providing context, and even suggesting fixes, error messages have become more human-friendly and helpful for Python beginners. Python 3.11 takes error messaging one step further to improve the debugging experience and expose an API for code analysis tools.
Sometimes, a single line of code can contain multiple instructions or a complex expression, which would be hard to debug in older Python versions:
$ cat test.py
x, y, z = 1, 2, 0
w = x / y / z
$ python3.10 test.py
Traceback (most recent call last):
File "/home/realpython/test.py", line 2, in <module>
w = x / y / z
ZeroDivisionError: float division by zero
$ python3.11a5 test.py
File "/home/realpython/test.py", line 2, in <module>
w = x / y / z
~~~~~~^~~
ZeroDivisionError: float division by zero
Here, one of the variables causes a zero division error. Python 3.10 tells you about the problem, but it doesn’t indicate the culprit. In Python 3.11, the traceback will include visual feedback about the exact location on a line that raised an exception. You’ll also have a programmatic way of getting that same information for tools.
Note that some of these enhanced tracebacks won’t work for code evaluated on the fly in the Python REPL, because the tracebacks require pre-compiled bytecode to keep track of the source lines:
Python 3.11.0a5 (main, Mar 1 2022, 10:05:02) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x, y, z = 1, 2, 0
>>> w = x / y / z
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: float division by zero
For more information on error locations in tracebacks, see PEP 657.
PEP 654: Exception Groups and except*
Python 3.11 will bring a new exception type called the exception group, which will allow for combining several exceptions into one container. The primary purpose of exception groups is to simplify error handling in concurrent code, especially Async IO, which has traditionally been verbose and messy. But there are other use cases, like grouping multiple exceptions raised during data validation.
Because the exception group is going to be a standard exception, you’ll still be able to handle it using the regular try … except clause. At the same time, there will be a new except* syntax for cherry-picking a particular member of an exception group while re-raising other members that you don’t currently want to deal with. If you catch a regular exception with except*, then Python will bundle it in a temporary exception group for you.
Note: Don’t confuse except* and *args, which look similar but have different meanings due to the distinct star placement.
Here’s a somewhat contrived example of an exception group in action:
>>> try:
... raise ExceptionGroup(
... "Validation exceptions", (
... ValueError("Invalid email"),
... TypeError("Age must be a number"),
... KeyError("No such country code"),
... )
... )
... except* (ValueError, TypeError) as subgroup:
... print(subgroup.exceptions)
... except* KeyError as subgroup:
... print(subgroup.exceptions)
...
(ValueError('Invalid email'), TypeError('Age must be a number'))
(KeyError('No such country code'),)
Unlike a regular except clause, the new syntax, except*, doesn’t stop when it finds a matching exception type but continues matching onward. This lets you handle more than one exception at a time. Another difference is that you end up with a subgroup of filtered exceptions that you can access through the .exceptions attribute.
Exception groups will have nice, hierarchical tracebacks, too:
+ Exception Group Traceback (most recent call last):
| File "/home/realpython/test.py", line 2, in <module>
| raise ExceptionGroup(
| ^^^^^^^^^^^^^^^^^^^^^
| ExceptionGroup: Validation exceptions
+-+---------------- 1 ----------------
| ValueError: Invalid email
+---------------- 2 ----------------
| TypeError: Age must be a number
+---------------- 3 ----------------
| KeyError: 'No such country code'
+------------------------------------
For more information on exception groups and except*, see PEP 654.
PEP 673: Self Type
Even though the release notes of Python 3.11 Alpha 5 mention this new feature, it hasn’t actually been merged to the main branch before tagging the release point in the repository’s history. If you’d like to get a taste of the Self type anyway, then use the latest commit in the main branch of CPython.
In short, you can use Self as a type hint, for example to annotate a method returning an instance of the class in which it was defined. There are many special methods in Python that return self. Apart from that, implementing fluent interface or data structures with cross-references, such as a tree or a linked list, would also benefit from this new type hint.
Below is a more concrete example that illustrates the usefulness of Self. In particular, due to the circular reference problem, you can’t annotate an attribute or a method with the class that you’re defining:
>>> from dataclasses import dataclass
>>> @dataclass
... class TreeNode:
... parent: TreeNode
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 3, in TreeNode
NameError: name 'TreeNode' is not defined
Until now, there were two work-arounds for this error:
- Use a string literal like
"TreeNode", which most type checkers recognize - Define a type variable with the base class as the bound
Both methods share the same drawback of having to update the code when your class name changes. In Python 3.11, you’ll have another, more elegant, meaningful, and name-independent way to communicate the same intent:
Python 3.11.0a5+ (heads/main:b35603532b, Mar 3 2022, 19:32:06) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from dataclasses import dataclass
>>> from typing import Self
>>> @dataclass
... class TreeNode:
... parent: Self
...
>>> TreeNode.__annotations__
{'parent': typing.Self}
For more information on the Self type, see PEP 673.
PEP 646: Variadic Generics
Again, although the Python 3.11 Alpha 5 release notes prominently feature variadic generics, they were still in very early development at the time of this article’s publication. The relevant code hasn’t even been merged into the main branch of the CPython repository. That said, if you insist on getting your hands dirty, then check out the draft implementation residing in a forked repository.
Note: This code won’t be useful until external type-checking tools and code editors catch up with it, because the Python interpreter ignores type hints at runtime.
Generic types, or generics for short, help achieve better type safety. They’re a way of specifying types parameterized with other types, which might be concrete types like int or str:
fruits: set[str] = {"apple", "banana", "orange"}
Such a declaration restricts the type of set elements to strings so that a type checker would consider any attempt to add any other type an error. You can also denote the generic parameters with abstract symbols representing placeholders for those types. In this case, you’ve been able to define a custom type variable in earlier Python versions with TypeVar, like so:
from typing import TypeVar
T = TypeVar("T", int, float, str)
def add(a: T, b: T) -> T:
return a + b
This would enforce that add() be called with arguments of the same type, T, which must be int, float, or str. A type checker would reject an attempt to call add() with two incompatible types.
Now, Python 3.11 will come with variadic generics in the form of TypeVarTuple, which targets a very specific use case found in scientific libraries like NumPy and dealing with multi-dimensional arrays. With variadic generics, you’ll be able to define the shape of an array by parameterizing its type with a variable number of placeholder types:
from typing import Generic, TypeVarTuple
Ts = TypeVarTuple("Ts")
class DatabaseTable(Generic[*Ts]):
def insert(self, row: tuple[*Ts]) -> None:
...
users: DatabaseTable[int, str, str] = DatabaseTable()
users.insert((1, "Joe", "Doe"))
users.insert((2, "Jane", "Doe"))
roles: DatabaseTable[str, str] = DatabaseTable()
users.insert(("a2099b0f-c614-4d8d-a195-0330b919ff7b", "user"))
users.insert(("ea35ce1f-2a0f-48bc-bf4a-c555a6a63c4f", "admin"))
For more information on variadic generics, see PEP 646.
Performance Optimizations
Python 3.11 will become noticeably faster. You can experience the difference for yourself by running a short code snippet in the terminal using the old and the new interpreter. Here’s a quick benchmark of calculating the thirty-fifth element of the Fibonacci sequence, deliberately implemented with recursion to simulate a challenging task for the computer:
$ SETUP='fib = lambda n: 1 if n < 2 else fib(n - 1) + fib(n - 2)'
$ python3.10 -m timeit -s "$SETUP" 'fib(35)'
1 loop, best of 5: 3.16 sec per loop
$ python3.11a5 -m timeit -s "$SETUP" 'fib(35)'
1 loop, best of 5: 1.96 sec per loop
As you can see, it takes slightly over three seconds for Python 3.10 to complete the computation and less than two seconds for Python 3.11 Alpha 5. That’s a whopping increase in performance! Naturally, the difference will vary depending on the task at hand, and you should generally expect less impressive numbers on average.
For more information on the performance optimizations, see the faster-cpython project on GitHub.
TOML Coming to Python 3.11
Until the feature freeze in Python 3.11 Beta 1, planned for May 2022, new features are continually discussed and added to the development branch of Python. The steering council has recently accepted PEP 680, which justifies the need for a TOML parser in Python’s standard library due to PEP 517’s move toward a new build system based on pyproject.toml. Previously, packaging tools had to bundle a library to read project metadata from this file.
The implementation will be based on the existing tomli library, which is actively maintained and well tested. Many popular tools already use it, which will make the switch smoother in those cases. However, the built-in module will be named tomllib.
Note: This feature is still in early development and isn’t available in the Python 3.11 Alpha 5 release.
With this addition, Python will gain the ability to read XML, JSON, and TOML out of the box. It’s too bad there won’t be any built-in support for YAML, though. Note that the current plan involves including a TOML parser only, without the corresponding serializer. Writing data in this format will still require installing and importing an external library.
Python Issues Moving to GitHub
Have you found a bug in Python 3.11 Alpha 5 and wondered where to report it? You can currently report bugs in Python’s Bug Tracker (BPO), hosted on Roundup. Following the migration of the CPython repository in February 2017, Roundup is one of the last remaining parts of Python’s infrastructure that hasn’t yet moved to GitHub. However, that’s going to change soon as the migration efforts accelerate.
Here’s what the current bug tracker for Python looks like:
In mid-February, the CPython Developer in Residence, Łukasz Langa, announced on the official Python Community forum that the steering council asked him to take over project management on the bug tracker’s migration. He also outlined the road map and asked for feedback during the testing period, highlighting that BPO would remain in read-only mode going forward. New issues will eventually be created through GitHub only.
If you’d like to read about the rationale behind this decision, then head over to PEP 581. In short, all other projects under the Python Software Foundation (PSF) already use GitHub Issues for bug tracking. Other than that, GitHub is well-known and actively maintained, and it has lots of interesting features missing from the current bug tracking system.
The plan is to migrate all existing tickets from BPO to GitHub before PyCon US 2022 in late April.
PyCon US 2022 Schedule Announced
After two years in a row of virtual PyCon US events due to the pandemic, it seems that PyCon US 2022 will finally take place in person, which many PyConistas have been looking forward to. Unless there are some unexpected developments, the conference will, for the first time, take place in Salt Lake City, Utah. At the same time, there will be an online streaming option available.
In late February, the organizers shared news about finalizing the conference schedule, which outlines an intense week full of tutorials, workshops, talks, and more:
| Dates | Events | Description |
|---|---|---|
| April 27-28 | Tutorials | Hands-on classes led by experienced instructors |
| April 28 | Sponsor Workshops | Usually a showcase of tools and solutions developed by the conference sponsors |
| April 28 | Education Summit | A joint discussion for teachers and educators focused on developing coding literacy through Python |
| April 29 | Maintainers Summit | A discussion around best practices on how to maintain and develop projects and communities |
| April 30 | Mentored Sprints for Diverse Beginners | A practical lesson on how to become an open-source project contributor for complete beginners |
| April 30 | PyLadies Auction | A charity auction with an opportunity for live bidding |
| April 29 - May 1 | Conference Talks | Five simultaneous presentation tracks, along with open spaces, sponsors, lightning talks, posters, dinners, and more |
| May 1 | Job Fair | An opportunity to apply for a job directly with sponsors at their booths |
| May 2-3 | Sprints | Group exercises to work on various open-source projects related to Python |
Note that tutorials have a limited number of spots, which fill up quickly, and require a $150 entry fee per session. The PyLadies Auction also has a $35 entry fee with an optional $15 contribution.
Are you going to attend PyCon US in Salt Lake City this year? Let everyone know in the comments section below. The Real Python team will be waiting for you there, ready to offer high fives and cool swag!
PSF Hiring to Improve PyPI
The Python Software Foundation (PSF) has announced that they’re looking to hire two contract developers to help build new features in the Python Package Index (PyPI), which is the official repository of Python packages. This decision comes after conducting a few surveys of the Python community, which identified key user requirements currently missing from PyPI.
The most desired feature turned out to be organization accounts in PyPI, which the two contractors will design, develop, and deploy. The goal is to allow for setting up an organization account in PyPI, inviting other users to join the organization, grouping those users into teams, and managing ownership and permissions across multiple projects. This would be a paid service offered to companies.
One of the contractor roles will focus on the backend, with the other handling the frontend. Both positions are remote-friendly, and the contract work is expected to start in early April 2022. The budget for each role is up to US $98 thousand for approximately 560 hours, with 35 hours per workweek.
Check out the evaluation criteria for more information.
Python 2.7 Removed From macOS
Technically, this is news from January, but it deserves mention here because it was a historic moment.
Operating systems derived from Unix, such as macOS and various Linux distributions, ship with Python as a dependency of numerous built-in software packages. However, unlike many Linux distributions, Apple hasn’t bothered to update the interpreter pre-installed in macOS since the release of Python 2.7 in 2010, even after 2.7 reached its end of life in April 2020.
That was not only a major security risk, which the UK Cybersecurity Agency (NCSC) warned about, but also an inconvenience for macOS users. They would often accidentally install packages into their legacy system-level interpreter instead of the desired virtual environment. Eventually, the rumors proved true when Apple quietly mentioned the removal of Python 2.7 from macOS in the release notes of macOS Monterey 12.3 Beta 5.
It’s interesting, though, that they decided to make such a big move in the middle of the operating system’s release cycle, through a beta version of the macOS update.
What’s Next for Python?
What’s your favorite piece of Python news from February? Did we miss anything notable? Let us know in the comments, and we might feature you in next month’s Python news roundup.
Happy Pythoning!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
