Python News

Python News: What's New From July 2021?

by David Amos Aug 16, 2021 community

July 2021 was an exciting month for the Python community! The Python Software Foundation hired the first-ever CPython Developer-in-Residence—a full-time paid position devoted to CPython development. In other news from the CPython developer team, tracebacks and error messages got some much-needed attention.

Let’s dive into the biggest Python news from the past month!

CPython Has a Full-Time Developer-in-Residence

In our June news roundup, we featured the Python Software Foundation’s announcement that they were hiring a CPython Developer-in-Residence. In July, the PSF’s plans came to fruition with the hiring of Łukasz Langa.

Łukasz, a CPython core developer and active member of the Python community, may be familiar to Real Python readers. In Episode 7 of the Real Python Podcast, Łukasz joined host Chris Bailey to talk about the origins of the Black code formatter, his experience as the Python release manager for Python 3.8 and 3.9, and how he melds Python with his interest in music.

As the first CPython Developer-in-Residence, Łukasz is responsible for:

  • Addressing pull requests and issue backlog
  • Performing analytical research to understand volunteer hours and funding for CPython
  • Investigating project priorities and their tasks going forward
  • Working on project priorities

In Łukasz’s statement about his new role, he describes his reaction to the announcement that the PSF was hiring:

When the PSF first announced the Developer in Residence position, I was immediately incredibly hopeful for Python. I think it’s a role with transformational potential for the project. In short, I believe the mission of the Developer in Residence (DIR) is to accelerate the developer experience of everybody else. This includes not only the core development team, but most importantly the drive-by contributors submitting pull requests and creating issues on the tracker. (Source)

Łukasz maintains a log of his work each week in a series of weekly reports on his personal website. During his first week on the job, he closed fourteen issues and fifty-four pull requests (PRs), reviewed nine PRs, and authored six of his own PRs.

“Don’t get too excited though about those numbers,” Łukasz writes in his first weekly report. “The way CPython is developed, many changes start on the main branch, and then get back ported to [Python] 3.10 and often also to 3.9. So some changes are tripled in those stats.”

The transparency that the weekly reports offer is refreshing and provides a unique look behind the scenes of the role. Future applicants will have a fantastic resource to help them understand what the job entails, what is working well, and where improvements can be made.

Łukasz wrote two weekly reports in July:

The series has continued into August. Each report includes a Highlights section featuring a couple of the most interesting challenges Łukasz worked on during the week. This section, in particular, is interesting to read because it goes into some depth about language features and bug fixes.

Congratulations to Łukasz on becoming the first CPython Developer-in-Residence! Real Python is excited to see what he accomplishes during his tenure and is ecstatic that the PSF successfully created and filled the role.

Python 3.11 Gets Enhanced Error Reporting

The release of Python 3.10 is just around the corner, and, as we reported in May, the new interpreter will be getting several improvements to error messages. The work on improved errors continues in Python 3.11.

Yes, that’s right! Even though Python 3.10 won’t be released until October, work on Python 3.11 is already under way!

Fine-Grained Error Locations in Tracebacks

Pablo Galindo, the release manager for Python 3.10 and 3.11, shared in a July 16, 2021 tweet that he and the team had finished implementing PEP 657. The PEP adds support for “fine grained error locations in tracebacks” and is a significant user experience upgrade for new and experienced Python developers alike.

To illustrate just how fine-grained the new error location reporting is, consider the following code snippet, which assigns the value 1 to the key "d" in a nested dictionary named x:

x["a"]["b"]["c"]["d"] = 1

In any Python 3 version up to Python 3.10, if any of the values for the keys "a", "b", or "c" are None, then executing the above snippet raises a TypeError telling you that you can’t subscript a NoneType:

Traceback (most recent call last):
  File "test.py", line 2, in <module>
    x['a']['b']['c']['d'] = 1
TypeError: 'NoneType' object is not subscriptable

This error is accurate, but it isn’t helpful. Which value is None? Is it the value at x["a"], x["b"], or x["c"]? Finding the exact location of the error requires more debugging and can be costly and time-consuming.

In Python 3.11, the same code snippet produces a traceback with some helpful annotations that point to exactly where the None value is located:

Traceback (most recent call last):
  File "test.py", line 2, in <module>
    x['a']['b']['c']['d'] = 1
    ~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable

The caret characters point to the exact location of the NoneType. The culprit is x["c"]! No more guessing, no more debugging. The error message by itself gives you all of the information you need to locate the cause of the error.

The Python community welcomed this change with enormous applause. Pablo’s tweet has garnered over four thousand likes at the time of writing this article, and the comments are filled with Python developers expressing their gratitude.

A few developers managed to find edge cases that aren’t currently supported—at least not in the current implementation. For example, Will McGugan wondered whether or not the new location reporting would work as expected for Asian characters and emojis. This Twitter thread confirmed the lack of support.

There is a cost to the change, too. As noted in the PEP, the implementation requires “adding new data to every bytecode instruction.”

The net result of the added bytecode instructions is a 22% increase in the size of the standard library’s .pyc files. That sounds like a significant increase, but it only amounts to about 6MB, and the team responsible for the PEP believes that:

[T]his is a very acceptable number since the order of magnitude of the overhead is very small, especially considering the storage size and memory capabilities of modern computers…

We understand that the extra cost of this information may not be acceptable for some users, so we propose an opt-out mechanism which will cause generated code objects to not have the extra information while also allowing pyc [sic] files to not include the extra information. (Source)

The opt-out mechanism consists of a new PYTHONDEBUGRANGES environment variable as well as a new command line option.

You can read PEP 657 for more information on the new error location reporting. You can find more examples of this feature in action in the What’s New In Python 3.11 document.

Improved Error Messages for Circular Imports

CPython Developer-in-Residence Łukasz Langa reported in his weekly report for July 19–26 that an improved error message for circular imports had been added to Python 3.11.

Consider the following package structure:

a
├── b
│   ├── c.py
│   └── __init__.py
└── __init__.py

Inside a/b/__init__.py is the following line of code:

import a.b.c

The c.py file contains this line of code:

import a.b

This gives rise to a situation where package b depends on module c even though module c also depends on package b. In Python versions up to Python 3.10, this structure generates a cryptic error message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/cpymain/a/b/__init__.py", line 1, in <module>
    import a.b.c
  File "/private/tmp/cpymain/a/b/c.py", line 3, in <module>
    a.b
AttributeError: module 'a' has no attribute 'b'

Messages like this one have frustrated innumerable Python developers!

Thanks to a pull request from CPython developer Filipe Laíns, the new error message is much clearer:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/cpymain/a/b/__init__.py", line 1, in <module>
    import a.b.c
    ^^^^^^^^^^^^
  File "/private/tmp/cpymain/a/b/c.py", line 3, in <module>
    a.b
    ^^^
AttributeError: cannot access submodule 'b' of module 'a'
(most likely due to a circular import)

Has any other parenthetical had the potential to save so much head bashing?

It’s not clear at the time of writing whether or not this change has been backported to Python 3.10. You may have to wait through one more release cycle to see the new error message.

What’s Next for Python?

July saw some exciting developments in Python. At Real Python, we’re excited about Python’s future and can’t wait to see what new things are in store for us in August.

What’s your favorite piece of Python news from July? Did we miss anything notable? Let us know in the comments, and we might feature you in next month’s Python news roundup.

Happy Pythoning!

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About David Amos

David Amos David Amos

David is a writer, programmer, and mathematician passionate about exploring mathematics through code.

» More about David

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Join us and get access to hundreds of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Join us and get access to hundreds of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Real Python Comment Policy: The most useful comments are those written with the goal of learning from or helping out other readers—after reading the whole article and all the earlier comments. Complaints and insults generally won’t make the cut here.

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Keep Learning

Related Tutorial Categories: community