Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Structuring a Python Application (Summary)

Christopher Trudeau
Structuring a Python Application Christopher Trudeau 04:39
Recommended Tutorial Course Slides (.pdf) Sample Code (.zip) Ask a Question

You’ve seen example layouts for a number of different application types:

  • One-off Python scripts
  • Installable single packages
  • Larger applications with internal packages
  • Django web applications
  • Flask web applications

You now have the tools to successfully prevent coder’s block by building out your application structure so that you’re not staring at a blank canvas trying to figure out where to start. Because Python is largely non-opinionated when it comes to application layouts, you can customize these example layouts to your heart’s content to better fit your use case.

These examples are neither hard-and-fast rules nor the only way to structure your application. With practice, you’ll develop the ability to build and customize your own useful Python application layouts!

Download

Sample Code (.zip)

 29.8 KB
Download

Course Slides (.pdf)

 1.1 MB

00:00 This series of videos has talked about the file structure for your Python applications. I started off by showing you a single-file program and associated configuration files, moved up to larger applications that used modules, and then went to larger applications still that used multiple modules.

00:17 Then, I showed you the differences in structures for web frameworks such as Django and Flask. In this lesson, I’m going to wrap up and then show you some variations.

00:26 As I mentioned in the first lesson, this video is based on an article by one of my colleagues at Real Python named Kyle. As a result of this, I’ve been able to see the comments attached to the article so I can possibly anticipate some of your questions. Python really isn’t opinionated about file structure. As a result, there’s a lot of variation out there.

00:45 Desktop applications have their own intricacies. If you’re doing Tk or Qt, they both require to have modules structured in a certain way, and that will impact the layout. Packaging has never been Python’s strength; it’s always been handled through third-party libraries rather than in the built-in ones.

01:03 That means there’s been lots of opinions and lots of change over the years. There are currently a half dozen popular virtual env management tools out there, so I don’t expect this is going to stay static.

01:16 It’s going to keep changing. Online, there’s currently a bit of a debate about whether or not you should put your Python source code inside of an src/ directory.

01:25 There’s pros and cons to this. The current directory that you’re in is sys.path. This makes it really easy to import something in a simple script, but it also makes it easy to make mistakes when you’re packaging.

01:36 Putting things inside of a src/ (source) directory forces you to test the installment and make sure that pieces are working properly. Unfortunately, some IDEs have problems with this structure, so it can make your life a little more complicated depending on what tools you’re using. And, the Python packaging authority does not currently use this mechanism. The debate rages on though. Throughout the lessons, you’ve seen me using setup.py. In addition to setup.py, there’s something called setup.cfg. setup.py is an actual Python program—setup.cfg is just a data file.

02:08 You can specify a lot of the same type of configuration in a CFG file as you can inside of setup.py. The file format uses an INI style declaration and it’s an easier way of providing defaults, so if your installer of choice wants to be able to give the person doing the installation some choices, using a text file to provide those choices and then allowing them to be overwritten is easier. And although CFG is supported, it doesn’t seem to be as popular as setup.py, but that might just be because I tend to spend a lot of my time looking at older code. Another variation is where to put the tests/ folder.

02:44 Some programmers prefer to put this inside of the module directory instead of the project route. This works fine enough. If you don’t want to include your tests inside of your package object, it means you have to update your manifest files in order to make sure that they don’t get bundled.

02:59 One of the reasons to put it inside of your module files is to have your unit tests in with the modules, and then if you have separate integration tests, have that out at the project directory.

03:08 You’ll see some of these variations out there as you go through other people’s code. What about the virtual environment? Where should you put it? Well, the short answer is: it kind of depends. This is mostly a personal preference, but the tools that you’re using may have different expectations.

03:23 The Real Python Pipenv guide talks about virtual environments and how to create them and how to use them, if you’re looking for more information.

03:31 One of the reasons to stick with fairly standard file structures is it makes it easier for you to use tools out there that have expectations. tox is used for testing your program against multiple versions of Python and has expectations about your file structure and setup.py. twine is for uploading your package to pypi.org, if you’re sharing it, and it expects the distribution wheel created by setup.py.

03:52 For large programs, it’s usually helpful to do a little bit of linting and formatting. There’s plenty of choice for linters and checkers out there: pyflakes, pylink, pychecker, pep8, and flake8 are all very good tools.

04:05 Something that’s become popular in the last little while is a tool called black. It reformats your Python code. Generally, this stops your teams from fighting over what code should look like.

04:16 Typically, the result is no one’s happy about what the code looks like, and that’s sort of the sign of a good compromise. There are also good tools out there to help you bootstrap your programs using scaffolding technology. Two common ones are pyscaffold and cookiecutter. That’s everything I wanted to say.

04:32 Hopefully, the content’s been useful for you. Thank you for your attention.

Avatar image for Denis Roy

Denis Roy on April 30, 2020

Here are clickable links of the references made in the video:

Different tooling has different expectations:

realpython.com/pipenv-guide

Project Scaffolding:

pyscaffold:

pyscaffold.org

cookiecutter:

cookiecutter.readthedocs.io

Avatar image for Lokman

Lokman on April 30, 2020

Thanks for the short video. Now I understand the world making packages.

Avatar image for mikesult

mikesult on May 1, 2020

Thanks Christopher, this was really useful for me to gain a better understanding of Application Structure at many different complexity levels.

Also thanks Denis Roy for the clickable links to the resources.

Avatar image for SamR

SamR on May 14, 2020

Very helpful information, thank you!

Avatar image for Alan ODannel

Alan ODannel on June 16, 2020

Nice video. This gave me a better understanding on Python’s packaging options.

Avatar image for jamesbrown68

jamesbrown68 on Aug. 10, 2020

Thank you, Christopher. This lesson was still a bit above my level, but I learned some things. You’ve got a nice, relaxed speaking voice and you clearly know your material.

Avatar image for Christopher Trudeau

Christopher Trudeau RP Team on Aug. 10, 2020

Glad you got something out of it James.

Avatar image for Ghani

Ghani on Oct. 9, 2020

Great course; thanks!

Avatar image for vaklaf

vaklaf on Jan. 12, 2022

Very helpful course for me! Thank you, Christopher!

Avatar image for Jon David

Jon David on May 16, 2022

Whew! That was fast! (but helpful)

Avatar image for Nicolas Gibaud

Nicolas Gibaud on April 12, 2023

Hello, great course ! It was really interesting. I’m just surprised that you don’t mention Poetry which is really related to the python packaging subject. What do you think of it ?

Avatar image for Christopher Trudeau

Christopher Trudeau RP Team on April 13, 2023

Hi Nicholas,

The packaging space in Python is evolving quickly and there are lots of tools both by the Python Packaging Authority and 3rd parties as well. I haven’t used Poetry enough to be an expert on it, my understanding is it does dependency management exceedingly well.

Most of the projects I work on lately are small enough, or have few enough developers, that I tend to favor simpler tools. Not to disparage the others, it is more a return-on-investment type calculation. As your projects get larger and/or when you’re dealing with larger teams, the overhead of some of the other approaches pays out.

We have a course in the queue that talks in detail about building Python packages. It includes a short lesson at the end giving a high level overview of both Flit and Poetry. Keep your eyes open for it.

Become a Member to join the conversation.