Python News: What's New From September 2022

Python 3.11.0rc2 Was Released

Every month, Python releases several versions from its different development branches. New releases typically add new features, fix bugs, correct security vulnerabilities, and more. September 2022 brought several new releases for Python programmers to test, use, and enjoy. Most notable was Python’s last 3.11 release candidate.

Python 3.11.0rc2 was released on Monday, September 12, 2022. This is the last preview version before the final release of Python 3.11.0:

Only reviewed changes that fix bugs are allowed during the release candidate phase. There will be very few, if any, code changes between this release candidate and the final release. As listed in the release post, the new features of the 3.11 series, as compared to 3.10, include the following:

• PEP 657 – Include Fine-Grained Error Locations in Tracebacks
• PEP 654 – Exception Groups and except*
• PEP 680 – tomllib: Support for Parsing TOML in the Standard Library
• PEP 673 – Self Type
• PEP 646 – Variadic Generics
• PEP 675 – Arbitrary Literal String Type
• PEP 655 – Marking individual TypedDict items as required or potentially-missing
• PEP 681 – Data Class Transforms

Python 3.11 also comes with a couple of other exciting updates. gh-90908 introduces task groups to asyncio, and gh-34627 allows for atomic grouping ((?>…)) and possessive quantifiers (*+, ++, ?+, {m,n}+) in regular expressions.

Plus, Python 3.11 is going to deliver faster performance:

The Faster CPython Project is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython for details. (Source)

To dive deeper into some of these cool new features of Python 3.11, check out the following resources depending on your specific needs and interests:

• Python 3.11 Preview: Even Better Error Messages
• Python 3.11 Preview: Task and Exception Groups
• Python 3.11 Preview: TOML and tomllib
• Python and TOML: New Best Friends

The first three tutorials in this list are part of a series of articles to help you get up and running with Python 3.11.

If you want to install this new release and try out some of its most exciting features, then check out the Real Python guide called How Can You Install a Pre-Release Version of Python?

Python 3.11.0 Release Was Postponed Until October 24

The Python core development team postponed the final release of Python 3.11.0 due to a week’s delay in the last release candidate, 3.11.0rc2. Now the official release is scheduled for Monday, October 24, 2022.

This final release was initially scheduled for Monday, October 3, 2022. So, we’ll have to wait three more weeks to welcome Python 3.11.0 onto our computers.

According to the 3.11 lifespan notes, this release will receive bugfix updates approximately every two months for about eighteen months.

Python Introduced a Breaking Change to Fix a Vulnerability

Python releases 3.10.7, 3.9.14, 3.8.14, and 3.7.14 are now available. Python 3.10, the latest stable version, released its seventh bugfix version out of schedule. This decision aimed to address a vulnerability that would allow denial of service (DoS) attacks due to the algorithmic complexity of str to int conversions.

The CVE platform registered this publicly disclosed cybersecurity vulnerability in its CVE-2020-10735 report. The original vulnerability description states:

A flaw was found in Python. In algorithms with quadratic time complexity using non-binary bases, when using int("text"), a system could take 50ms to parse an int string with 100,000 digits and 5s for 1,000,000 digits (float, decimal, int.from_bytes(), and int() for binary bases 2, 4, 8, 16, and 32 are not affected). The highest threat from this vulnerability is to system availability. (Source)

Meanwhile, the What’s New In Python 3.10 page in the Python documentation describes the issue as follows:

Converting between int and str in bases other than 2 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now raises a ValueError if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. (Source)

This change will break existing code that runs this type of conversion on numbers that exceed a given number of digits. The default limit for the number of digits is now 4300 digits. Here’s a short example that uncovers the breaking change by running a str to int conversion in Python 3.10.6 vs 3.10.7:

>>> # Python 3.10.6
>>> int("2" * 5432)
222222222222222222222222222222222222222222222222222222222222222...

>>> # Python 3.10.7
>>> int("2" * 5432)
Traceback (most recent call last):
   ...
ValueError: Exceeds the limit (4300) for integer string conversion:
   value has 5432 digits; use sys.set_int_max_str_digits()
   to increase the limit.

This call to int() works fine in Python 3.10.6 and raises a ValueError in Python 3.10.7. Note that Python can still work with large integers. The error is only raised when converting between integers and strings. This new behavior could break more than one codebase out there, so keep an eye on it if your code usually deals with this kind of conversion.

Fortunately, you can increase the limit for the allowed number of digits when you expect an operation to exceed it. To do this, you can use one of the following:

• The PYTHONINTMAXSTRDIGITS environment variable
• The -X int_max_str_digits command-line flag
• The set_int_max_str_digits() function from the sys module

Check the documentation for more details on changing the default limit if you expect your code to exceed this value.

Finally, the 3.9.14, 3.8.14, and 3.7.14 security releases also address the described issue, along with some less urgent security improvements and fixes. So, upgrading your installation is highly recommended if you’re using any of these Python series in production code.

New Releases in the Python Ecosystem

The global Python community never stops pushing the Python ecosystem into the future. As usual, you’ll find plenty of new releases from different libraries, frameworks, and projects. Django, pandas, TensorFlow, and Matplotlib are some of the most visible projects in September’s list of new releases.

What’s Next for Python?

So, what’s your favorite piece of Python news from September? Did we miss anything notable? Are you going to give Python 3.11.0rc2 a try? What do you think about the breaking changes around the str to int conversion in Python 3.10.7? Let us know in the comments!

Happy Pythoning!