In January 2022, the code formatter Black saw its first non-beta release and published a new stability policy. IPython, the powerful interactive Python shell, marked the release of version 8.0, its first major version release in three years. Additionally, PEP 665, aimed at making reproducible installs easier by specifying a format for lock files, was rejected. Last but not least, a fifteen-year-old memory leak bug in Python was fixed.
Let’s dive into the biggest Python news stories from the past month!
Free Bonus: Click here to get a Python Cheat Sheet and learn the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.
Black No Longer Beta
The developers of Black, an opinionated code formatter, are now confident enough to call the latest release stable. This announcement brings Black out of beta for the first time:
Code formatting can be the source of a surprising amount of conflict among developers. This is why code formatters, or linters, help enforce style conventions to maintain consistency across a whole codebase. Linters suggest changes, while code formatters rewrite your code:
This makes your codebase more consistent, helps catch errors early, and makes code easier to scan.
YAPF is an example of a formatter. It comes with the PEP 8 style guide as a default, but it’s not strongly opinionated, giving you a lot of control over its configuration.
Black goes further: it comes with a PEP 8 compliant style, but on the whole, it’s not configurable. The idea behind disallowing configuration is that you free up your brain to focus on the actual code by relinquishing control over style. Many believe this restriction gives them much more freedom to be creative coders. But of course, not everyone likes to give up this control!
One crucial feature of opinionated formatters like Black is that they make your diffs much more informative. If you’ve ever committed a cleanup or formatting commit to your version control system, you may have inadvertently polluted your diff.
With a configurable formatter, or no formatter, changing or enforcing style midway through a project can make functional code changes hard to track. This is because a diff will bring up stylistic changes alongside functional code changes, which can be time-consuming to disentangle.
If you use Black throughout the entire project, though, you can be pretty confident that your diff will only display functional code changes.
With the first stable release of Black comes a new stability policy, which aims to make explicit what you can expect from the stable release of Black.
The stability policy mainly focuses on how formatting changes will be handled. Python is evolving, and occasionally it makes sense for Black to make a small change to their style. The stability policy will prevent Black from making such changes within a given calendar year.
The stability policy pledges that any code formatted with Black in a given January will produce exactly the same output in December of that year, regardless of any internal changes to Black. The version number of Black is also linked to the year, so version 22 corresponds to Black during the year 2022. When 2023 comes around, Black will release version 23. If there are any style changes, they’ll be applied in version 23, and not before.
Black is already quite popular, and this new stable release will likely increase its adoption.
Release of IPython 8.0
This month saw the release of IPython 8.0, three years after the release of IPython 7.0. This new version reflects a massive collaborative effort incorporating over 250 pull requests. The main highlights are default Black formatting, improved tracebacks, and improved suggestions.
IPython was created by Fernando Pérez in the early 2000s, supposedly while he was procrastinating on his university work. The core of IPython is an interactive shell, which can be used as a replacement for Python’s default REPL. It includes handy features such as syntax highlighting, autosuggestion, and code completion:
IPython is used as the Python kernel for Jupyter Notebook, a program that displays code execution in a document-like format. Scientists and educators often use it to display Python code with elegant outputs or rich-text formatting in a more broadly familiar format:
For more on Jupyter’s capabilities, check out Introduction to Jupyter Notebook.
From a developer’s perspective, one of the most significant efforts in developing the newest release was a focus on making the codebase leaner. This effort paid off, reducing the codebase from 37,500 lines of code across 348 files to 36,100 lines of code across 294 files!
A summary of lessons learned was also published with the release, and it contains some words of wisdom for developers.
PEP 665 - Rejected
PEP 665 was rejected by the steering council this month. This somewhat controversial PEP aimed to make reproducible installations easier with Python.
Most package managers have what’s referred to as a lock file. One such file in the Python ecosystem is the poetry.lock file, which you’ll find in Poetry. Pipenv also has its own lock file.
You can think of a lock file as a far more detailed requirements.txt file. It goes into more detail about the exact files, versions, and even installer used for installation. A lock file facilitates reproducible setups across environments, large teams, or user bases. PEP 665 aimed to standardize a lock file format.
For a more detailed discussion, check out Real Python Podcast Episode 93 with the PEP’s author, Python core developer Brett Cannon. This podcast was recorded before the PEP was rejected.
According to its critics, this PEP fell short by requiring that all packages be distributed as wheels and by leaving some doubt about how exactly to handle sdist (source distribution) packages. For more detailed information on wheels and sdists, check out What Are Python Wheels and Why Should You Care?
On the podcast, Brett Cannon called Python “the glue code of the world,” thanks to Python’s ability to incorporate libraries from other languages, such as C, Go, and Rust. When installing a package dependent on other languages, Python has a lot of work to do to make sure everything is set up properly. This work isn’t always successful, which has caused some frustration for the Python community.
PEP 665 was written to tackle the trickiness of installation and the pitfalls of having slightly different installations on different systems. Though it was rejected, there’s still a lot of work being done on this problem. Some related PEPs are:
- PEP 643 - Metadata for Package Source Distributions
- PEP 621 - Storing project metadata in
pyproject.toml - PEP 650 - Specifying Installer Requirements for Python Projects
We at Real Python want to give special thanks to Brett Cannon, who was also an author on PEP 621 and PEP 650, for his continuing efforts in this area.
For now, at least, Python will not standardize a lock file format, and tools such as Poetry and Pipenv will continue having their own versions.
Would you like to see a standardized lock file? Have you had problems with reproducing installs? We’d love to hear your opinions in the comments below!
Fifteen-Year-Old Bug, Fixed!
In a remarkable display of tenacity, Victor Stinner marked a fifteen-year-old bug as “fixed”.
This bug was raised in 2007 by Ben Sizer, to show that Python had a memory leak upon exit. That is, Python wouldn’t free up all the memory that it had been using. Like how you should close a file object when working with files, programs should free up memory once they don’t need it anymore.
Since this is a relatively common issue with all sorts of programs, operating systems usually perform their own cleanup after an app has exited. The auto cleanup meant that this memory leak bug was relatively low-impact and didn’t affect most users. However, it was a problem for programs that started up many Python subprocesses, as demonstrated by the demo C program that was posted in the bug report:
#include <Python.h>
int main(int argc, char *argv[])
{
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
Py_Initialize(); Py_Finalize();
}
This program starts and ends Python multiple times, resulting in a growing number of unreleased references, which represents a memory leak. To learn more about memory and memory management, check out Memory Management in Python.
These leaks were wide-ranging, coming from several different sources. It’s taken numerous modifications and fifteen years (!) to reduce the bug to a level which merited marking it as “fixed.” There’s still some memory leakage, but it’s good enough for now. Thank you, Victor Stinner!
What’s Next for Python?
What’s your favorite piece of Python news from January? Did we miss anything notable? Let us know in the comments, and we might feature you in next month’s Python news roundup.
Happy Pythoning!
