As February takes a rare leap forward with an extra day this year, the Python community followed suit!
Python versions 3.12 and 3.11 receive a security fix, and CPython source distributions now document the software supply chain to allow for a more effective vulnerability detection. Another Rust-based tool makes its way into the Python ecosystem, promising exciting improvements to the existing package management system.
Looking ahead, the reveal of the PyCon US 2024 schedule gives us a glimpse into the upcoming Python conference. In other news, the Python Software Foundation launches recurring Office Hours to enhance community support in the Grants Program.
Let’s dive into the biggest Python news from the past month!
Python 3.12 and 3.11 Receive a Security Fix
The Python 3.12.2 and Python 3.11.8 patch versions were released, incorporating hundreds of commits and a host of bug fixes. Aside from that, they both provide a small security fix to an obscure feature of Python that allows for arbitrary code execution.
In a nutshell, this new security fix forbids the processing of hidden path configuration files (.pth) located in a virtual environment’s site-packages/ folder:
venv/
│
├── bin/
│
├── include/
│
├── lib/
│ │
│ └── python3.12/
│ │
│ └── site-packages/
│ │
│ └── .your-hidden.pth
│
├── lib64/
│
└── pyvenv.cfg
On a Unix-like operating system, any file becomes implicitly hidden when its name starts with a leading dot. On Windows, a file needs the corresponding attribute set to be hidden. Note that the directory structure presented above might look slightly different on Windows.
Path configuration files are plain text files that the site module in the Python standard library automatically parses and processes upon the interpreter startup. Historically, these files helped facilitate editable installs and implement hooks into the importing machinery. They essentially let you append extra folders to the Python search path, which is accessible through the sys.path variable.
Unfortunately, .pth files have a quirk that makes it possible to execute any code on startup:
venv/lib/python3.12/site-packages/.your-hidden.pth
import os; print("This will run on Python startup!")
When a line starts with the word import, it’s interpreted as literal Python code. Therefore, one could technically inject malicious code this way. To make matters worse, Python versions prior to those with the mentioned security fix would execute such code from hidden .pth files, preventing the user from inspecting the code and assessing the risk properly:
When you receive a handful of files, you, as a cautious person, check their contents before executing. If Python source files are hidden, it’s okay, because you saw that nothing suspicious is imported in the files that you execute. But pth files can be executed even if you do not see them and there are no references in visible files. (Source)
In the official announcement, it was promised that this fix would roll out to earlier Python versions, including Python 3.8, Python 3.9, and Python 3.10, shortly afterward.
SBOM Documents Become Available for CPython
Continuing on the security theme, Python 3.12.2 marks the beginning of the inclusion of Software Bill-of-Materials (SBOM) documents for CPython going forward. It’s the result of ongoing efforts to improve the security vulnerability management strategy for Python.
SBOM documents represent the inventory of all the components, modules, and libraries that make up CPython, including their versions and the relationships between them. They’re a valuable resource facilitating better security and compliance management. By detailing what’s inside the source releases of CPython, SBOMs help identify potential security vulnerabilities and licensing issues impacting the use and distribution of the software.
The SBOM document detailing CPython’s bundled dependencies uses the JSON flavor of the Software Package Data Exchange (SPDX) format. You can find it in the CPython source code on GitHub by navigating to Misc/sbom.spdx.json. Alternatively, you can grab SBOM files corresponding to the downloadable artifacts directly from the release page of the given Python version.
Only source distributions of Python have their SBOMs at the moment. However, future releases will also include SBOMs of other artifacts, such as Python installers for various platforms that may have additional dependencies.
For context, you can review the underlying GitHub issue opened by Seth Larson, who’s the Security Developer-in-Residence hired by the PSF to take responsibility for Python’s security improvements.
Astral Unveils Python Packaging in Rust
Astral, the company founded by Charlie Marsh, who gained fame after authoring Ruff—a Rust-based code linter and formatter for Python—has unveiled another tool named uv. It stands for universal, reflecting uv’s broad applicability planned for the future. The tool is being advertised as a Cargo for Python, aiming to become a drop-in replacement for existing dependency resolvers like pip, pip-tools, and more.
Note: Interestingly, Astral has also announced they’ll be taking over Rye, which is another Rust-based tool. Rye started as an experimental project by Armin Ronacher, most notably known as the creator of Flask.
Recognizing the alignment of their visions, Astral and Armin have agreed to collaborate, with Astral taking stewardship of Rye. The ultimate goal is to evolve uv into a unified project and package management tool, building on the initial groundwork laid by Rye.
Since uv is implemented in Rust, it doesn’t depend on Python. Unless you need to compile packages from source, uv is blazingly fast. According to official benchmarks, uv can be between ten and over a hundred times faster than its predecessors. In addition, uv has a familiar interface, making it straightforward to adopt, and it produces arguably more user-friendly and informative output than pip.
Once you install uv, for example, with pipx, you can create virtual environments and install dependencies into them using these commands:
$ uv venv
Using Python 3.12.2 interpreter at: /home/user/.pyenv/versions/3.12.2/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
$ uv pip install jupyterlab
Resolved 88 packages in 796ms
Downloaded 88 packages in 6.92s
Installed 88 packages in 78ms
+ anyio==4.3.0
+ argon2-cffi==23.1.0
⋮
+ websocket-client==1.7.0
Each subcommand represents an isolated namespace corresponding to the tool in question. For example, uv venv mimics the interface of Python’s venv module to some extent, while uv pip shares many command-line arguments with the Python packaging tool pip. Stay tuned, as more subcommands are on their way!
Note: Activating a virtual environment isn’t strictly necessary for uv because it can detect a local .venv/ subfolder and use it automatically. At the same time, installing dependencies into the global Python interpreter isn’t possible with uv at the moment.
Despite being in the early stages of development, uv has already seen rapid growth with developers actively resolving GitHub issues, sometimes within minutes, and implementing new features in quick succession. This responsiveness, coupled with the frequent releases, is the hallmark of the Astral team, who follow the same principles as with their previous work on the Ruff project.
While uv undoubtedly brings many benefits and has generally been well received by the Python community, it’s also stirred some controversy. Critics argue that it adds to the already existing pool of competing tools like pip, pip-tools, pipx, Pipenv, Poetry, PDM, Hatch, and so on. This may remind you of this classic webcomic by xkcd:
Moreover, the decision to build a commercial company around tools based on decades of work from open-source projects hasn’t gone down well with some in the Python community. Concerns have also been raised about the potential negative impact on those original tools and their future. Finally, Rust might present a high entry barrier for Python developers who’d like to contribute to or debug uv, Ruff, and others in this new wave of tools.
PyCon US 2024 Schedule Announced
PyCon US, one of the most anticipated events in the Python community, has announced its full schedule and keynote speaker lineup for 2024. The conference will take place this May in Pittsburgh and will be chaired by Mariatta Wijaya for the second consecutive year.
The announcement revealed a record-breaking number of talk proposals received for this year’s conference, totaling nearly one thousand! The organizers were grateful to the volunteers who devoted their time and expertise to reviewing and selecting the talk proposals. Their contribution has helped shape an exciting and diverse agenda covering a broad range of topics.
In response to ongoing health and safety concerns, masks will unfortunately remain mandatory for attendees at the in-person conference. However, for those who cannot attend in person or prefer to participate remotely, an online attendance option is also available. As always, recordings of the talks will be uploaded to the PyCon US channel on YouTube shortly after the event.
Be sure to check out How to Get the Most Out of PyCon US if you plan to attend the conference and want to have a great experience.
The PSF Introduces Grants Program Office Hours
The Python Software Foundation (PSF) is a community-driven, non-profit entity that oversees the development and promotion of the Python programming language and its ecosystem. One of the foundation’s responsibilities includes supporting Python-centric events and projects through a grants program sponsored by donations from organizations and individuals.
These grants help partially finance costs like venue rental, catering, or speaker fees at local meetups, workshops, and conferences. Travel grants often provide help to community members who otherwise couldn’t afford to attend PyCon conferences and engage with the wider Python community. In recent years, the PSF has awarded hundreds of such grants.
After last year’s turmoil surrounding DjangoCon Africa’s grant application, the PSF promised to conduct an internal retrospective of the grant review process and take steps to improve it. In February this year, they announced the introduction of a recurring grants program office hours as part of that initiative. Its main goal is to establish a direct line of communication with the Python community:
The PSF Grants Program Office Hours is a text-only chat based office hour hosted on the Python Software Foundation Discord at 2-3PM UTC (9AM Eastern) on the third Tuesday of the month. (Source)
The PSF invites anyone interested in the grants program to join and ask questions, share links to their grant application drafts, or to provide feedback and help shape the program’s future.
The event will be hosted by the PSF staff members. Initially, they’ll be Laura Graves, Senior Accountant, and Marie Nordin, Community Communications Manager.
What’s Next for Python?
February 2024 was a busy month. From the safety enhancements in the latest Python patch releases to the introduction of SBOM documents for CPython, it’s clear that security is a top priority. The entry of a new Rust-based tool into the Python packaging space is a significant development that may change how Python developers manage dependencies and virtual environments.
The anticipation for PyCon US 2024 is building up. With the schedule and keynote speakers announced, this year’s event is promising a wealth of knowledge and networking opportunities for Python enthusiasts. Also, the PSF’s introduction of Grants Program Office Hours is a commendable step towards greater transparency and community engagement.
What’s your favorite Python news story from February? Let us know in the comments. Happy Pythoning!
