Python News: What's New From August 2022

Python Has PEPs, NumPy Has NEPs, pandas Now Has PDEPs

The first pandas enhancement proposal (PDEP) was submitted on August 3, 2022, and was entitled Purpose and guidelines. Enhancement proposals aren’t new to the Python community. Python has had PEPs since 2000, and NumPy followed with NEPs in 2017.

The first PDEP follows in the tradition of PEPs and NEPs, with PDEP-1 being an introduction to the idea behind the enhancement proposals themselves.

In a nutshell, PDEPs are intended to aid the proposal process for major changes to pandas, such as moving a module from the main pandas repository to a break-off repository. PDEPs are not for quick fixes, but for major undertakings that involve the wider community and, more often than not, some significant trade-offs.

Working out complex issues isn’t ideally suited to a thread-based medium, like GitHub issues. It can be hard for a discussion to stay focused if anyone can respond at any time, even if the original idea is a good one.

GitHub issue threads can create noise not only for the core developers, but for contributors and end users too. Additionally, they can bury good but complex ideas by not providing an appropriate medium for discussing them. Contributor h-vetinari raised this topic in a GitHub issue in 2019:

The more intricate the API implications, the harder it is to discuss in GitHub comments (because there is usually too many things to consider at the same time, or the comments/threads get ridiculously long or both). That does not mean that the given change does not have merit though, just that it’s (likely) too hard to discuss in a thread format. (Source)

The GitHub issue that h-vetinari raised three years ago has now been closed with the pull request for PDEP-1. This may lay a blueprint for the PDEP life cycle going forward. PDEPs will probably get started when someone creates an issue. If the issue is recognized as being significant and valuable, then the person who raised it may be directed to create a PDEP.

This move to PDEPs means that the roadmap that’s typically used for communicating larger changes to pandas will slowly migrate toward PDEPs.

How do you feel about the move to PDEPs? Share your thoughts in the comments!

Python Ecosystem Celebrates New Releases

The Python community didn’t rest throughout August, even though it’s typically a month for vacations. As usual, there have been plenty of releases in the Python ecosystem. From CPython to CircuitPython, there are plenty of new features for you to start playing with. Read on for a selection of releases and milestones.

CPython

The CPython team is still gearing up for the release of Python 3.11 in October 2022. If you’re interested in learning more about the 3.11 release, check out some in-depth Real Python tutorials exploring the new 3.11 features, such as exception groups, tomllib, and better error messages.

In August, Python 3.11.0rc1, a release candidate, was released:

If you want to help out the Python community at large, then test this new release candidate by running your code and packages with it. Apart from the noted deprecations, most things should still work, and you’ll hopefully notice a nice speed boost as well.

If you want a guide, then check out the Real Python tutorial on how to install a pre-release version of Python!

If you find something in the 3.11 release candidate that you think might be a bug, check out the issue board to see if it’s being discussed. If you really think you’ve found a bug that no one else has found, open a new issue!

In preparation for the release of Python 3.11, NumPy has gotten ahead of the pack and published wheels for 3.11:

This is great news for a lot of other packages that depend on NumPy and wouldn’t be able to get started on porting to 3.11 without the a NumPy 3.11 wheel.

While 3.11 might be all the rage right now, 3.10 hasn’t been forgotten. This month, a new maintenance release came out:

You don’t need to upgrade to 3.10.6 from 3.10.x unless it’s convenient to do so. That said, upgrading should’t break your existing code on 3.10.x, unless a regression has been unintentionally introduced, which is unlikely.

Let us know what you think of the new release in the comments below! Which feature are you most excited about?

Read the Docs

This month, Read the Docs celebrated twelve years since its first commit:

A humble requirements file would lead to one of the most well-known packages in the Python ecosystem.

Read the Docs will help you create and distribute your documentation to your users. Not only does it develop a package to automate the creation of your online documentation, but it’ll also host your documentation for free. Read the Docs serves over 55 million pages of documentation a month, which translates to a tidy 40TB of bandwidth.

To keep up-to-date with Read the Docs, check out its blog. Also, Read the Docs is collecting entries for its awesome documentation projects repository—check that out for some inspiration for your documentation.

Python Extends Lead at Top of TIOBE Index

The August edition of the TIOBE Programming Community index saw Python gain another 2 percent market share, continuing to hold the top spot with its highest-ever market share:

Python seems to be unstoppable. (Source)

It continues to hold over C and Java, which have been neck and neck since TIOBE started the ranking back in 2001.

It’s important to note, however, that the TIOBE index is based on how web pages dedicated to the language rank in different searches. So the TIOBE ranking isn’t about the best language, or the most popular. It mainly indicates the amount of content about that language that’s ranking on the Internet. Still, Python for the win!

PyPI Fights Malware Attacks

In August, some PyPI users were subject to the first known phishing campaign targeting PyPI:

Users have reported receiving an email asking them to validate their packages to prevent removal from PyPI. The message contains links that redirect to a fake version of the PyPI login page.

If you were to insert your username and password into this fake login page, your credentials would be sent to the people behind the phishing campaign. They would then use the account credentials to log into the real PyPI, tamper with your uploaded packages, and potentially lock you out.

So far, any credentials that the attackers have managed to obtain have been used to upload malware-infected versions of the user’s packages. So, the next time someone downloads the latest version of a compromised package, their computer would be infected with that malware.

PyPI has published the address of the fake site as sites[dot]google[dot]com/view/pypivalidate, and the credential information is sent to linkedopports[dot]com. Using these malicious addresses, PyPI has found various compromised packages.

Any compromised package found has been promptly cleaned up and temporarily frozen. The packages are frozen to lock out the attacker while PyPI restores access to the rightful owner and to ensure that users can continue to download the packages without fear of malware.

Additionally, using the linkedopports[dot]com pattern, PyPI has uncovered many typosquatting packages. Typosquatting in PyPI involves uploading a malware package with a name that’s very similar to a popular package. For instance, when downloading a package with python -m pip install requests, you might commonly mistype requests as reqyests. A typosquatter might upload a malware package as reqyests to infect those who make this typo.

Earlier in the month, Snyk, a developer security platform, had discovered various smaller packages uploaded to PyPI with malware that steals Discord and Roblox credentials from Windows users. The packages use PyInstaller to obfuscate the malicious code contained within them.

So, even with the recent move to two-factor authentication (2FA), it goes to show that there are no silver bullets when it comes to security. You should always be aware that when using pip, you’re downloading code from the Internet to run on your machine. Do your best to make sure the package is legit—for instance, you can check a package’s readme for indications that it’s real:

• Does it have documentation?
• Does it have a community around it?
• Is the source code publicly available?
• Does the package have a website with contact information?

These are just some indicators that a legitimate package might have, but as mentioned, there’s no silver bullet, and you need to keep your wits about you when navigating the modern interconnected world!

In a timely fashion, TalkPython released a podcast episode this month with Dustin Ingram, PyPI maintainer and PSF director, where they chat about Python packaging, PyPI, security, and the recent move to 2FA.

Have you encountered any malware or suspicious packages in PyPI? Reach out to security@pypi.org with all the details. Remember, PyPI is an open source platform funded by sponsors and users. They don’t make any profits, and they need the support of the community to keep everyone safe.

What’s Next for Python

What were you most excited about in the Python world in August? Did we miss anything? Will you be trying out any of the new releases? Do you have any thoughts about pandas’ move to PDEPs? What do you think about the recent security issues around PyPI? Let us know in the comments!

Happy Pythoning!