In August 2023, Python 3.12.0rc1 came out! With several exciting features, improvements, and optimizations, this release is only two steps away from the final release scheduled for October. If you want to stay on the cutting edge, then you must give it a try. But note that you shouldn’t use it in production.
Another exciting release was Python in Excel, which allows you to leverage the power of Python inside your Excel workbooks. You’ll be able to use Python’s data science ecosystem while you remain in your Excel comfort zone with known formulas, charts, and more.
But that’s not all! The Python Software Foundation (PSF) announced a new roster of fellows and a safety and security engineer for PyPI. Some key package maintainers were busy with the Python DataFrame Summit 2023, and several key libraries released new versions.
Let’s dive into the most exciting Python news from August 2023!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
Python 3.12.0 Release Candidate Arrives
This August, Python put out its first release candidate version, 3.12.0rc1. This version is only two steps away from the final release, 3.12.0, which is scheduled for October 2. Before that, Python will deliver another release candidate, planned for September 4.
As Python ventures into the release candidate stage, only confirmed bug fixes will be accepted into the codebase. The goal is to have as few code changes as possible. Most likely, the core team will focus on:
- Polishing and documenting all the changes
- Updating the What’s New document
As you can read in the release notes, 3.12 will have the following list of new features compared to Python 3.11:
- More flexible f-string parsing, allowing you to do more with your f-strings than you previously could (PEP 701)
- Support for the buffer protocol in Python code (PEP 688)
- A new debugging/profiling API (PEP 669)
- Support for isolated subinterpreters with separate global interpreter locks (PEP 684)
- Even more improved error messages, meaning that you get suggestions for more exceptions potentially caused by typos
- Support for the Linux
perfprofiler to report Python function names in traces - Many large and small performance improvements, such as PEP 709, which deliver an estimated 5 percent overall performance improvement
If you’d like to learn more about some of these improvements, then check out Real Python’s previews of more intuitive and consistent f-strings, ever better error messages, and support for the Linux perf profiler.
As usual, this version also brings several deprecations that you may need to consider. For a detailed list of changes, additions, and removals, you can check the changelog document.
Just like other pre-release versions, Python 3.12.0rc1 is intended for experimentation and testing purposes only and isn’t recommended for use in production.
Python Makes Its Way Into Microsoft Excel
On August 22, Microsoft announced Python in Excel, a new and exciting feature that combines the flexibility of Excel and the power of Python. This combination may have a considerable impact on the data science industry.
This is a huge announcement, and even Guido van Rossum himself has been helping with the integration of both tools:
Python in Excel allows you to natively use Python inside an Excel workbook without any additional setup requirements. You only need the new PY function, which lets you input Python code directly into Excel cells:
The Python code will run in Microsoft Cloud, and you’ll get the results directly on your worksheet.
This new feature enables you to widely extend Excel’s functionalities with Python’s tools for data analysis and manipulation. You’ll be able to quickly perform tasks like the following:
- Cleaning your data with Python tools
- Creating advanced plots and visualizations to present your data
- Running machine learning models
- Making predictions and forecasting
Again, you’ll be able to use Python’s rich ecosystem for data science in your Excel sheets and also continue using all the usual features that you already know in Excel, such as formulas, charts, and PivotTables.
To get Python into Excel, Microsoft partnered with Anaconda, which provides a large Python repository and the actual Python distribution that powers this Excel feature. Your Python code will run in Azure with enterprise-level security. This ensures that you’ll have safe access to the large number of Python libraries that Anaconda offers.
You won’t have to worry about installing Python runtimes, libraries, and dependencies. All of that will be available to you out of the box. You’ll also be able to work on a team and share your Python in Excel workbooks with confidence.
If you’re interested in trying Python in Excel, then you need to know that the feature is gradually rolling out for public preview to those who are in the Microsoft 365 Insider program’s Beta Channel.
If you decide to join the Insider program to try out Python in Excel, then go ahead and install the latest Insider build of Excel, open a blank workbook, and take the following steps:
- Select Formulas in the ribbon.
- Select Insert Python.
- Select the Try Preview button in the dialog box that appears.
To get you started, Microsoft has put together a Getting Started with Python in Excel guide that you can follow while exploring this new feature.
Future improvements for Python in Excel will impact key areas related to user experience. These areas include auto-complete and syntax highlighting, enhanced error behaviors, and help and documentation support.
Python Software Foundation (PSF) Shares Great News
The Python Software Foundation has also been active this past month. They made a few exciting announcements that thrilled the Python community.
PyPI has a new safety and security engineer, while the PSF published their list of fellows for the first quarter of 2023 and the Annual Impact Report for 2022.
Safety and Security Engineer for PyPI
On August 4, the PSF announced that Mike Fiedler has joined the team for the next year as the PyPI safety and security engineer. Mike is a dedicated member of the Python packaging community and became a PyPI maintainer in 2022.
Note: To hear from Mike Fiedler about the progress that he’s made in his first months on the job, check out his interview on the Real Python Podcast.
He’s going to be tackling a lot of important tasks at PyPI. The expected outcomes of his work include:
- Increased support for package maintainers, including multi-maintainer projects
- Improved reporting infrastructure for malicious projects
- Reduced response time for malware reports and account recovery requests
Amazon Web Services (AWS) will be funding Mike’s position for the next year. On his side, Mike has already been reporting some exciting improvements on PyPI. Now the platform has the following new capabilities:
Starting on August 8, newly registered users must enable 2FA before they can perform any management actions on PyPI. This new requirement is part of the PSF effort to improve the overall security of PyPI.
Back in 2021, PyPI and GitHub completed the secret scanning integration for all PyPI users. This integration makes it possible to notify PyPI if a user accidentally pushes a PyPI token to a GitHub public repository. In response to this security risk, PyPI automatically revokes the token to prevent any misuse.
Here’s how GitHub described the initiative:
Users with secret scanning enabled on their free public repositories will now receive alerts for any potential secrets exposed in an issue’s title, description, or comments, including historical revisions. (Source)
This greatly improves GitHub’s secret scanning capabilities, which were previously limited to code. As a result, PyPI will have enhanced token-revoking capabilities and improved security for its users.
PSF Fellows for the First Quarter of 2023
Also in August, the PSF announced the first batch of PSF fellows for 2023! These are people who make amazing contributions to the Python community. Here they are, along with where you can find them online:
- Abhishek Mishra: Twitter, LinkedIn, Website
- Barney Gale: GitHub, LinkedIn
- Eric Traut: GitHub, LinkedIn
- Gina Häußge: Mastodon, GitHub, Website
- Grishma Jena: Twitter, LinkedIn, Website, Instagram
- Samuel Colvin: Twitter, GitHub
- Saptak Sengupta: Mastodon, GitHub, Website
- Soon Seng Goh
These PSF members have significantly contributed to the Python ecosystem by serving as extraordinary leaders, growing the Python scientific community, and maintaining libraries. They’ve also worked on creating educational content, organizing events and conferences, running local community organizations, mentoring Python learners, and more.
PSF Annual Impact Report for 2022
The PSF Annual Impact Report for 2022 was also published in August. The report provides a summary of the work that the PSF did over the year. It also shares the organization’s experiences and goals.
In this report, Deb Nicholson described her experiences as the executive director of the PSF:
I joined as the new Executive Director in April of 2022 and am still learning about all the amazing things people do with Python and meeting more of the inspiring people that make up our vast and vibrant community.
She also talked about PyCon US 2022, which brought together 1,753 in-person and 761 online attendees last year. Finally, she outlined the PSF goals for the coming year:
In the coming year, we’ll be working to deepen our connections with Python leaders around the world, expanding our grant outreach, and supporting the D&I workgroup and creating structures that help the PSF and Pythonistas around the world gain more insight into what we can do for each other.
We’ll be doing more to connect with those who are learning Python and providing resources for newer Pythonistas in classrooms, at meet-ups and every place where people are looking to learn. We’ll also be tackling big issues like overhauling security for the Python ecosystem, building out more support for community and corporate organizations using PyPI and continuing to invest in the sustainability of the core Python community.
Python packaging project manager Shamika Mohanan summarized the work that the PSF has been doing in order to improve the user experience on PyPI, while Łukasz Langa talked about his work as the inaugural developer in residence.
Python DataFrame Summit Gathers Package Maintainers
The Python DataFrame Summit 2023 happened in Basel, Switzerland, on August 12 and 13. This is a meeting of maintainers and users of different DataFrame technologies. They get together to discuss project road maps and ideas and to find synergies between the different projects. This year, the participants represented many different packages, including pandas, Polars, Dask, PySpark, and NumPy.
Over the weekend, the projects also made good progress as the core developers got to work together on new features. The new pandas v2.1.0 release includes some of this work. Similarly, Polars v0.18.14 and v0.18.15 contain new features discussed during the summit.
One exciting development that’s been in the works for some time is the Python DataFrame API Standard. The goal of this standard is to define a minimal API that allows you to write code that can handle any kind of DataFrame. In the newest versions of pandas and Polars, you can call .__dataframe_consortium_standard__() to get a standard-compliant DataFrame.
For a basic example, consider the following code to select a column in a DataFrame in both pandas and Polars:
>>> if isinstance(data, pd.DataFrame):
... column = data.loc[:, [column_name]]
... elif isinstance(data, pl.DataFrame):
... column = data.select(pl.col(column_name))
...
Here, you use an if block to treat data differently based on whether it’s a pandas or Polars DataFrame. For pandas, you use .loc[] to pick out the column, while for Polars, you need to use .select(). An alternative is now to convert the DataFrame to a standardized one. The following code works for both pandas and Polars without any if tests:
>>> data_std = data.__dataframe_consortium_standard__()
>>> data_std.get_columns_by_name([column_name])
A standard-compliant DataFrame supports .get_columns_by_name() independently of whether it originated in pandas or Polars. You can convert back to the original type by accessing .dataframe.
The Consortium for Python Data API Standards is still discussing and working on the Python DataFrame API Standard. You can learn more about the standard by watching the presentation DataFrame-agnostic code: are we there yet? by Marco Gorelli at EuroSciPy 2023.
New Releases of Python Packages Available
In August, the Python ecosystem bustled with activity, rolling out a series of exciting developments for essential Python projects. Some of the most relevant releases include:
Django 4.2.3 was experiencing a few issues that have now been corrected. An issue was causing the QuerySet.aggregate() function to crash with window functions. Another problem made your code crash when using a reference in a subquery for grouping. On top of that, there was an issue trying to group by multi-valued annotations in a query. Thankfully, these issues have been resolved, leading to a smoother user experience.
Pydantic also got updated with several bug fixes and corrections. Similarly, the Flask web framework and the command-line interface (CLI) solution Click both released upgraded versions.
The continuous evolution of these popular and useful tools showcases the dynamic nature of Python’s ecosystem, keeping developers eagerly anticipating upcoming innovations.
What’s Next for Python?
August 2023 was an exciting month for the Python community. We had the first release candidate of Python 3.12, plus the announcement of Python in Excel, a promising integration for the data science community.
The Python ecosystem continues to evolve and advance. The future of DataFrame technologies looks exciting. The community continues to release new versions of numerous projects, including Django, Pydantic, Flask, and Click. We can’t wait to see what’s next! Let us know your thoughts on these developments in the comments below.
Happy Pythoning!
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
