Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Publishing Python Packages to PyPI (Summary)

Christopher Trudeau
Publishing Python Packages to PyPI Christopher Trudeau 04:09
Recommended Tutorial Course Slides (PDF) Ask a Question

You now know how to prepare your project and upload it to PyPI, so that it can be installed and used by other people. While there are a few steps that you need to go through, seeing your own package on PyPI is a great payoff. Having others find your project useful is even better!

In this video course, you’ve learned:

  • Why packages and virtual environments exist
  • How to structure a package
  • How to use build systems
  • The contents of the pyproject.toml file
  • How to use the build and twine tools

You also looked at the alternative Flit and Poetry tools.

Links In This Course:

Further Investigation:

Similar Real Python Content:

Download

Course Slides (PDF)

 5.0 MB

00:00 In the previous lesson, I gave you a whirlwind tour of Flit and Poetry. This is the last lesson, where I’ll summarize the course and point you at some more resources. Not quite a rhyming couplet, but close. Better than a cheese pun, at least.

00:15 If there’s one thing you should take away from this course, it’s that packaging is complicated. The current best practice is to use a pyproject.toml file to define your package. Remember that the package and module name don’t have to be the same, and you often see prefixes on the module name to avoid name collisions in PyPI.

00:35 you can install a package in editable mode using pip install -e. Doing so means you can edit in place and have the changes be reflected in your virtual environment. The second theme from this course, besides cheese puns, is the details of the pyproject.toml file.

00:52 It contains all the project metadata information, your dependencies, the ability to define a script hook, and tool-specific sections.

01:03 A build system is what creates the files that you upload, and although setuptools is the most common one, there are others out there. What the build system produces is a wheel and source distribution package.

01:16 A wheel file is actually in the ZIP archive format with an expected structure on the inside. You use the build and twine tools to build and publish packages, respectively.

01:27 Or, if you prefer, there are other systems out there to help you do the same, like Flit and Poetry.

01:35 A whole lot of Python Enhancement Proposals have been written describing the packaging ecosystem. If you really want to dig deep, here’s a little light bedtime reading.

01:44 PEP 427 talks about the wheel format. 440 is about versions and dependency specifications. 508 is even more dependency stuff, while 517 is about separating the build system from the package structure, making it easier to have third-party build tools.

02:03 518 tells you what you need to be able to build a Python package. 621 is about the metadata info in the pyproject.toml file, and 660 specifies how editable installs should work for wheels.

02:20 If PEPs are a bit too heavy for you, there are other ways of getting at some of the same info. Dane Hillard’s excellent book Publishing Python Packages goes into great detail about the different parts of a package, what choices you have, and how all this integrates with the various tools in the packaging toolchain.

02:37 The Python Packaging Authority is a group separate from the core developers who make the recommendations around Python packaging. Many of the tools demonstrated in this course actually come from them.

02:48 These links are to their website, details on the specs for packaging, and a list of presentations that cover all things packaging. Because packaging is complex, there are lots of opinions out there on how things should change.

03:02 This article by Chris Warrick does a great summary of all the different options, what need each tool fulfills, and how he thinks things should improve. I believe I grumbled a bit about the src/ directory.

03:14 This is a good counter-article to my grumbling, explaining how to structure your packages to do good testing and why the src/ directory is a good idea.

03:24 For more Real Python content in related areas, you might be interested in Python Application Layouts: A Reference. This tutorial covers the structure of a package and the reasoning behind it.

03:36 For more on docs, especially using Sphinx and Read the Docs, this course as you covered. Or for testing, either of these can be helpful. And finally, if you want to learn the details behind the wheel format, this article is a great deep dive.

03:53 That’s all for this course. Thanks for your attention. I hope you weren’t feta up by the cheese jokes. And as is only appropriate, I leave you with the immortal words of Sir Michael Palin: We do have Camembert, sir. Oh, the cat’s eaten it.

Avatar image for Ramon Zuniga

Ramon Zuniga on July 12, 2023

Is it a good practice/idea to build packages for reusable data models throughout different projects?

Avatar image for Christopher Trudeau

Christopher Trudeau RP Team on July 15, 2023

Hi Ramon,

Sorry for the delayed reply, I was away on business. I’ve never done it myself, but don’t see a reason why not. Anything that you feel you’re re-using can be put in a package. Whether you publish it to PyPI or not might be a different question. The service is there and free, and there is nothing stopping you from putting stuff up there, but the spirit is that it is meant for re-use by others.

Become a Member to join the conversation.