The new year has arrived, and January brought a flurry of new and interesting Python enhancement proposals (PEPs). Topics range from f-string formalization and no-GIL Python to packaging. There’s been ample discussion on the Python Discourse forum about the implications of these PEPs, and if you’re still a bit wary of diving deeper into the discussions, then you can get a softer introduction here.
There have also been a couple of noteworthy new releases, first and foremost the fourth alpha release of Python 3.12. And finally, there’s a new job posting out for a security developer in residence sponsored by the Python Software Foundation (PSF).
Let’s dive into the biggest Python news from the past month!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
PEP 701 Attempts to Formalize f-Strings
Python’s f-strings are great, but there are currently a few edge cases that might make you scratch your head.
For example, maybe you wanted to greet the new year with a carefully constructed f-string, but you ran into unexpected troubles:
>>> f"happy {"\n".join(["twenty", "three", "🥳"]) # it's 2023!}!!"
File "<stdin>", line 1
f"happy {"\n".join(["twenty", "three", "🥳"]) # it's 2023!}!!"
^
SyntaxError: unexpected character after line continuation character
You used some double quotes (") inside the curly braces ({}). It looks like Python isn’t happy with that and considers the f-string closed, even though the quotes appear inside the f-string expression part.
The new year is still fresh, so you won’t let this get you down! You just change the double quotes to single quotes:
>>> f"Happy {'\n'.join(['twenty', 'three', '🥳']) # Yes! 2023!}!!"
File "<stdin>", line 1
f"Happy {'\n'.join(['twenty', 'three', '🥳']) # Yes! 2023!}!!"
^
SyntaxError: f-string expression part cannot include a backslash
Oh! There’s another issue with your new year’s greeting f-string. It looks like you can’t include the backslash (\) character in an f-string expression. Well, maybe 2023 is more about expanding in breadth than in depth, you think to yourself. So you replace the newline character (\n) with a whitespace character (' ') and try again:
>>> f"Happy {' '.join(['twenty', 'three', '🥳']) # Yes! 2023!}!!"
File "<stdin>", line 1
f"Happy {' '.join(['twenty', 'three', '🥳']) # Yes! 2023!}!!"
^
SyntaxError: f-string expression part cannot include '#'
Argh! Yet another problem! It seems like you can’t include comments in the f-string expression part—not even when they’re full of joy and optimism!
Python 3.6 introduced f-strings, and the LL(1) parser that Python used back then wasn’t able to handle these edge cases.
Maybe running into the limitations of f-strings has slightly dampened your optimistic beginning of 2023. But don’t write this year off quite yet, because these limitations may go away soon.
Pablo Galindo Salgado authored PEP 701 – Syntactic formalization of f-strings at the end of last year. The PEP proposes a formalized grammar for f-strings, which could be directly integrated into the parsing expression grammar (PEG) parser that Python’s been using since version 3.9.
If this PEP is accepted and implemented, then f-strings will support the edge cases shown above. The expression parts of an f-string will be able to contain:
- Strings that use the same type of quotes that the f-string literal uses
- Backslash characters (
\) - Newlines
- Comments written with the number sign (
#)
There are some potential disadvantages to these changes as well. They mainly concern the possibility of quote reuse, which you can read more about in the Discourse discussion thread.
If PEP 701 gets accepted, maybe you’ll be able to greet 2024 with a slight alteration of the f-string that you tried this year:
>>> "Happy {
... "\n".join(["twenty", "four", "🥳"]) # Yes! 2024!
... }!!"
'Happy twenty\nfour\n🥳!!'
Fingers crossed for some new year’s resolutions to come through and make their way into 2024. Will it be the new syntactic formalization of f-strings?
PEP 703 Proposes to Remove the Global Interpreter Lock (GIL)
Python’s global interpreter lock (GIL) has represented a performance bottleneck for applications that want to make use of modern multi-core central processing units (CPUs).
In a recent discussion, CPython developer in residence Łukasz Langa characterized its removal as a total game changer:
There have been prior attempts to remove the GIL from CPython, such as the Gilectomy proposed by Larry Hastings, which has since been abandoned.
This month, Sam Gross published PEP 703 – Making the Global Interpreter Lock Optional in CPython. His approach aims to make the GIL optional so that developers can avoid the issues that it presents:
Python’s global interpreter lock makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications.
[…] current workarounds for the GIL are “complex to maintain” and cause “lower developer productivity”. The GIL makes it more difficult to develop and maintain scientific and numeric computing libraries as well leading to library designs that are more difficult to use. (Source)
The PEP includes a fully functional prototype implementation, so you can try it out today!
Gross groups the changes to the CPython source code into four categories:
- Reference counting
- Memory management
- Container thread-safety
- Locking and atomic APIs
The implementation changes, as well as related considerations, are described in the PEP, which has turned out to be one of the longest PEPs published to date!
There’s been a lot of enthusiasm and support for the PEP. However, there are also voices in the community that are concerned about whether such a change might cause a fracturing of the Python community, similar to what happened during the transition from Python 2 to Python 3:
Some commenters call for a release of Python 4. Others worry about the added maintenance effort that they expect an alternative build will add to open-source package maintainers. Further, there’s some concern that no-GIL Python might slow down Python programs that don’t require releasing the GIL.
There’s a lively discussion going on in the Discourse thread, so feel free to join in and share your use cases, concerns, hopes, and opinions on this potentially groundbreaking PEP.
PEP 704 Suggests Package Installers to Promote Virtual Environments
Using virtual environments in Python is a suggested best practice—but new learners often don’t known about them, and the process of creating and activating a virtual environment adds additional learning overhead. Many beginners might therefore avoid working with virtual environments. However, installing packages into your global Python environment can cause problems!
To avoid system-wide package installations and help practitioners understand that they should use a virtual environment, CPython core developer and Python Packaging Authority (PyPA) member Pradyun Gedam wrote PEP 704.
This PEP suggests a few best practices related to virtual environments that package installers, such as pip, could implement:
- Require virtual environments by default.
- Install virtual environments in a default location.
PEP 704 suggests that package installers should quit with an error and print an informational message if a user attempts to install a package into the global environment:
$ python -m pip install black
ERROR: Could not find an activated virtualenv (required).
Please create and activate a virtual environment
before installing a Python package:
https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments
Note that all the text in this error message is just an example because the PEP doesn’t mention any specific wording.
In pip, this could be implemented by changing the existing environment variable PIP_REQUIRE_VENV to 1 and adapting the error message accordingly.
Additionally, PEP 704 suggests that the default location for a Python virtual environment should be in the directory tree in a folder named .venv. Until now, there hasn’t been a clear suggestion regarding how to name your virtual environment folder.
The intention of the PEP is to facilitate the use of virtual environments and standardize the install location and naming. Note that none of these changes would be enforced. Still, they could represent a shift toward more adoption of virtual environments, and a more standardized idea of where to find them.
If accepted, these changes could help to reduce the barrier to using and understanding virtual environments for newcomers to the Python language. However, there’s been a lively discussion in the associated Discourse forum thread, and it’s still up in the air whether the proposal will be accepted or not—and even whether or not it should be a PEP in the first place.
The Community Continues to Discuss Python Packaging
If reading the discussion about PEP 704 left you wanting more, then you can rejoice! There’s always more to discuss in the world of Python packaging.
Pradyun Gedam published a blog post summing up his thoughts on the Python packaging ecosystem, which adds perspective for people who aren’t deeply involved in the topic.
Also in the Python packaging space, Nathaniel J. Smith officially announced Pybi and Posy. These tools, which he’s been working on for a while, could allow an alternative workflow that packages Python itself together with any other packages that your project needs.
Overall, the conversation about improving packaging in Python has continued this January, with lots of new input!
The PSF Is Hiring a Security Developer in Residence
Malware attacks on the Python Package Index (PyPI) have increased over the past year, with a noteworthy incident happening last August. There have been prior attempts to increase security, for example by requiring two-factor authentication, but malicious packages on PyPI have still been cropping up.
Therefore, it’s a treat to read a recent announcement by the Python Software Foundation (PSF) about hiring a security developer in residence!
The job description is summed up on the PSF blog in the announcement:
The new role will be responsible for addressing security issues across PSF projects such as CPython and PyPI, and applying knowledge and expertise and working with volunteers to implement key improvements in a timely manner. They will also establish new processes and features that make it easier to prevent, detect, and respond to security risks to lay a foundation that makes it easier and more sustainable for the community to identify and address security issues going forward. (Source)
If you think you’re the right person for the job, then you can apply to the job posting. Or you can send the application link to your security-expert friend.
The Python Ecosystem Issues New Releases
You may have read in November’s news article that the Python release team made Python 3.12.0 alpha available. In January, right on schedule, they published the fourth alpha release.
Learn more about features and updates on the Python 3.12.0a4 release page or play around with it yourself after installing the pre-release version, then send your feedback to the team.
A handful of popular third-party libraries have also received new releases during the past month:
Check out their release pages to understand whether these updates are relevant for you. And if you’d like to acquaint yourself with a couple of these libraries, then check out the Django for Web Development learning path or the Image Processing With the Python Pillow Library tutorial here at Real Python.
What’s Next for Python?
So, what’s your favorite piece of Python news from January 2023? Did we miss anything notable? Are you in favor of the proposed approach to no-GIL Python? What do you think about attempts to improve Python packaging? And are you looking forward to infinitely nesting quote characters in f-strings? Let us know in the comments!
Happy Pythoning!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
