Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Structuring a Python Application (Summary)
You’ve seen example layouts for a number of different application types:
- One-off Python scripts
- Installable single packages
- Larger applications with internal packages
- Django web applications
- Flask web applications
You now have the tools to successfully prevent coder’s block by building out your application structure so that you’re not staring at a blank canvas trying to figure out where to start. Because Python is largely non-opinionated when it comes to application layouts, you can customize these example layouts to your heart’s content to better fit your use case.
These examples are neither hard-and-fast rules nor the only way to structure your application. With practice, you’ll develop the ability to build and customize your own useful Python application layouts!
Congratulations, you made it to the end of the course! What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment in the discussion section and let us know.
00:00 This series of videos has talked about the file structure for your Python applications. I started off by showing you a single-file program and associated configuration files, moved up to larger applications that used modules, and then went to larger applications still that used multiple modules.
00:17 Then, I showed you the differences in structures for web frameworks such as Django and Flask. In this lesson, I’m going to wrap up and then show you some variations.
00:26 As I mentioned in the first lesson, this video is based on an article by one of my colleagues at Real Python named Kyle. As a result of this, I’ve been able to see the comments attached to the article so I can possibly anticipate some of your questions. Python really isn’t opinionated about file structure. As a result, there’s a lot of variation out there.
00:45 Desktop applications have their own intricacies. If you’re doing Tk or Qt, they both require to have modules structured in a certain way, and that will impact the layout. Packaging has never been Python’s strength; it’s always been handled through third-party libraries rather than in the built-in ones.
01:03 That means there’s been lots of opinions and lots of change over the years. There are currently a half dozen popular virtual env management tools out there, so I don’t expect this is going to stay static.
01:16
It’s going to keep changing. Online, there’s currently a bit of a debate about whether or not you should put your Python source code inside of an src/ directory.
01:25
There’s pros and cons to this. The current directory that you’re in is sys.path. This makes it really easy to import something in a simple script, but it also makes it easy to make mistakes when you’re packaging.
01:36
Putting things inside of a src/ (source) directory forces you to test the installment and make sure that pieces are working properly. Unfortunately, some IDEs have problems with this structure, so it can make your life a little more complicated depending on what tools you’re using. And, the Python packaging authority does not currently use this mechanism. The debate rages on though. Throughout the lessons, you’ve seen me using setup.py. In addition to setup.py, there’s something called setup.cfg. setup.py is an actual Python program—setup.cfg is just a data file.
02:08
You can specify a lot of the same type of configuration in a CFG file as you can inside of setup.py. The file format uses an INI style declaration and it’s an easier way of providing defaults, so if your installer of choice wants to be able to give the person doing the installation some choices, using a text file to provide those choices and then allowing them to be overwritten is easier. And although CFG is supported, it doesn’t seem to be as popular as setup.py, but that might just be because I tend to spend a lot of my time looking at older code. Another variation is where to put the tests/ folder.
02:44 Some programmers prefer to put this inside of the module directory instead of the project route. This works fine enough. If you don’t want to include your tests inside of your package object, it means you have to update your manifest files in order to make sure that they don’t get bundled.
02:59 One of the reasons to put it inside of your module files is to have your unit tests in with the modules, and then if you have separate integration tests, have that out at the project directory.
03:08 You’ll see some of these variations out there as you go through other people’s code. What about the virtual environment? Where should you put it? Well, the short answer is: it kind of depends. This is mostly a personal preference, but the tools that you’re using may have different expectations.
03:23 The Real Python Pipenv guide talks about virtual environments and how to create them and how to use them, if you’re looking for more information.
03:31
One of the reasons to stick with fairly standard file structures is it makes it easier for you to use tools out there that have expectations. tox is used for testing your program against multiple versions of Python and has expectations about your file structure and setup.py. twine is for uploading your package to pypi.org, if you’re sharing it, and it expects the distribution wheel created by setup.py.
03:52
For large programs, it’s usually helpful to do a little bit of linting and formatting. There’s plenty of choice for linters and checkers out there: pyflakes, pylink, pychecker, pep8, and flake8 are all very good tools.
04:05
Something that’s become popular in the last little while is a tool called black. It reformats your Python code. Generally, this stops your teams from fighting over what code should look like.
04:16
Typically, the result is no one’s happy about what the code looks like, and that’s sort of the sign of a good compromise. There are also good tools out there to help you bootstrap your programs using scaffolding technology. Two common ones are pyscaffold and cookiecutter. That’s everything I wanted to say.
04:32 Hopefully, the content’s been useful for you. Thank you for your attention.
Lokman on April 30, 2020
Thanks for the short video. Now I understand the world making packages.
mikesult on May 1, 2020
Thanks Christopher, this was really useful for me to gain a better understanding of Application Structure at many different complexity levels.
Also thanks Denis Roy for the clickable links to the resources.
SamR on May 14, 2020
Very helpful information, thank you!
Alan ODannel on June 16, 2020
Nice video. This gave me a better understanding on Python’s packaging options.
jamesbrown68 on Aug. 10, 2020
Thank you, Christopher. This lesson was still a bit above my level, but I learned some things. You’ve got a nice, relaxed speaking voice and you clearly know your material.
Christopher Trudeau RP Team on Aug. 10, 2020
Glad you got something out of it James.
Ghani on Oct. 9, 2020
Great course; thanks!
vaklaf on Jan. 12, 2022
Very helpful course for me! Thank you, Christopher!
Jon David on May 16, 2022
Whew! That was fast! (but helpful)
Nicolas Gibaud on April 12, 2023
Hello, great course ! It was really interesting. I’m just surprised that you don’t mention Poetry which is really related to the python packaging subject. What do you think of it ?
Christopher Trudeau RP Team on April 13, 2023
Hi Nicholas,
The packaging space in Python is evolving quickly and there are lots of tools both by the Python Packaging Authority and 3rd parties as well. I haven’t used Poetry enough to be an expert on it, my understanding is it does dependency management exceedingly well.
Most of the projects I work on lately are small enough, or have few enough developers, that I tend to favor simpler tools. Not to disparage the others, it is more a return-on-investment type calculation. As your projects get larger and/or when you’re dealing with larger teams, the overhead of some of the other approaches pays out.
We have a course in the queue that talks in detail about building Python packages. It includes a short lesson at the end giving a high level overview of both Flit and Poetry. Keep your eyes open for it.
Become a Member to join the conversation.
Denis Roy on April 30, 2020
Here are clickable links of the references made in the video:
Different tooling has different expectations:
realpython.com/pipenv-guide
Project Scaffolding:
pyscaffold:
pyscaffold.org
cookiecutter:
cookiecutter.readthedocs.io