Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Requirements Files Best Practices
00:02 Now that you have a good idea about how requirements files work, and how you can leverage them to organize your Python dependencies, let’s talk about some best practices for structuring and organizing your requirements files.
00:14 When it comes to version specifiers, it’s a good idea to use exact versioning most of the time, this is also called pinning and it means that you lock down the exact version of a package that will be installed. Trust me, this will help you avoid headaches in the long run.
00:31 It’s also important that you include secondary dependencies in their exact version numbers. If you don’t include secondary dependencies and pin them down to exact versions, what can happen is that a secondary dependency gets silently upgraded on the next deploy and that might break your application.
00:48 So my recommendation here would be to use exact versions for all of your packages including secondary dependencies. Now it may make sense to go without a version specifier for some development dependencies, for example, if you’re using a third-party debugger, like ipdb or pdb++ it can make sense to leave that unpinned so that you can always work with the latest version.
01:13
Personally, I am leaning towards pinning as many packages as possible. From personal experience, I know that this can help avoid a lot of trouble. When it comes to naming your requirements files the most popular choice seems to be requirements.txt and then also requirements-dev.txt for those development dependencies.
01:36
Other popular choices for requirements files names, include requirements.pip or requirements.lock but really, this is just a naming convention and those files would function exactly the same as the requirements.txt files you’ve seen before.
01:52 Whatever you name your file, typically they would be placed in the root folder of your project. Some projects also create a separate requirements folder, but services like Heroku typically expect that the requirements file sits in the root folder of the project so if you go with this convention, it usually makes things a little bit easier for you because you won’t have to overwrite config variables and point them to a non standard location for your requirements files.
02:18
That is why I would recommend that you always put them into the root folder of your project. One more handy feature in requirements files is that they support comments, you can see this in the example here, all you need to do is place a hash (#) character, sort of like a Python comment, and then all text behind that is going to be ignored.
02:37
It’s a great idea to use comments in your requirements files, for example, you could use them to explain the difference between your requirements.txt file and the requirements-dev.txt file, this will be helpful for any developer working with the project in the future.
02:55 Or you could leave a comment on a specific dependency to explain your choice or to explain why a specific version is needed, adding comments to your requirements files is a great idea.
03:06
There is a common question around ordering dependencies inside requirements files, so should you order them, what order should you put them in, usually I start my requirements files with the direct dependencies, so these would be the dependencies that I initially installed using the pip install command, this would include packages like Requests, or Flask, and I like to sort them alphabetically because that makes it easier to find what I am looking for.
03:35 Then, after those direct dependencies I would put a section with secondary dependencies and I usually use comments to indicate the difference between the two.
03:44
So all of these secondary dependencies are dependencies that I didn’t install directly through pip install but they were installed because a direct dependency required them.
03:53
For example here, if you pip install Flask it will bring in the jinja2 templating library and also the werkzeug library.
04:01
So my typical workflow here would be to capture all dependencies using the pip freeze command and then go through them in an editor to split them into direct dependencies and secondary dependencies and perform the ordering.
04:14
When you do a pip install using that requirements file pip will figure out the right order to install your packages in so you don’t have to worry about breaking things by changing the order.
04:23 Another best practice that you already learned about is the development and production environment split. Pip allows you to link up requirements files so that when you install one, pip will go out and actually also install the other, there is a whole lesson for that in the course, so you might want to rewatch that, as it’s a highly recommended and very useful feature.
04:41 Let’s do a quick recap here, those were the requirements files best practices that you just learned, first, you learned about the importance of version pinning, second, we discussed naming schemes for requirements files, after that, you learned how to use comments in requirements files and then you learned how you can make your requirements files easier to understand by splitting the dependencies into first order and secondary dependencies.
05:04 And finally, we did a quick recap on the technique that allows you to split up your development and production dependencies. If you follow these conventions, and best practices, it will make it easier for other developers to work with your Python projects.
Dan Bader RP Team on June 4, 2020
There’s no equivalent to Python doc strings, but you can use comments in requirements files:
A line that begins with
#is treated as a comment and ignored. Whitespace followed by a#causes the#and the remainder of the line to be treated as a comment. (Source)
So you can do something like this:
# This is a comment
package1==0.0.1
package2>=0.0.1
Hope this helps you out! :)
Zarata on Oct. 19, 2020
This statement confuses me: “Pip allows you to link up requirements files so that when you install one, pip will go out and actually also install the other, there is a whole lesson for that in the course, so you might want to rewatch that, as it’s a highly recommended and very useful feature.” I did go back to the previous lesson and couldn’t immediately find what you reference. What I did find seemed to imply one manually chooses one or the other, not the automated link to get both the dev and prod installs.
Dan Bader RP Team on Oct. 20, 2020
Inside of a requirements file, you can use -r FILENAME to reference dependencies stored in another requirements file.
For example, let’s imagine this was your requirements-dev.txt file:
-r requirements.txt
package1==1.0.0
package2==1.2.0
package3==0.4.5
When you run pip install -r requirements-dev.txt the -r requirements.txt line in requirements-dev.txt tells pip to install the dependencies listed in requirements.txt first.
I hope that clears it up :) It’s a very useful feature.
Here’s the direct link to the relevant part of the transcript and lesson video:
realpython.com/lessons/separating-development-and-production-dependencies/#t=167.0
Become a Member to join the conversation.
RicardoSantana on June 4, 2020
Dan, can you use Doc Strings in the requirements.txt file? I would think there will be times where a greater degree of detail would be useful for the sake of having a good historical record. What is your recommendation regargind this?