In the previous lesson, I showed you how to use
pip and where it puts any installed packages. In this lesson, I’ll introduce you to using
pipenv to manage both your package installation and virtual environments.
In the previous lesson, you saw some of the problems of using
pip on its own without a virtual env. If you have multiple projects with conflicting dependencies, keeping everything in the same
site-packages directory is problematic.
A common way of managing your package versions is by including their information in a
requirements.txt file. This also has some problems. If you only include your dependencies and not your dependents’ dependents inside of your
requirements.txt file, your build won’t be deterministic.
Think back to the example of installing
requests in the previous lesson.
requests is dependent on
certifi, amongst other things.
certifi releases something new, your next installation will end up using a different set of libraries than your first installation. This is a good way for
production to be mismatched and can result in a “worked on my machine” kind of problem.
So instead of specifying only
requests in your
requirements.txt, you could redirect
pip freeze and grab all the dependencies like I showed you in the previous lesson. This solves the problem above, but also means you’re now responsible for all of the packages.
You might want to get that new version of
certifi because of a bug fix. Keeping everything in
requirements.txt means you’re managing all of it.
requirements.txt is one flat file, meaning it can only specify a single environment. What if your dev environment needs testing packages that your production environment doesn’t require?
I have seen some package maintainers have
dev-requirements files, which works. But now you’ve got twice the things you have to maintain. As you might’ve guessed, like a late-night TV commercial talking about all the problems of dull knives, I’ve got a sharper knife to sell you.
pipenv solves most of these problems.
If you hadn’t heard of it before,
pipenv is a third-party tool maintained by the same people who maintain the core Python libraries. it uses
virtualenv under the hood to give you that sharper knife.
You may recall from the previous lesson that I did some of the work in a directory called
without_pipenv. That is because
pipenv does some magic things if it finds a
requirements.txt file. So to avoid that, I’m going to create a new directory called
using_pipenv and stay inside of that one to avoid any side effects.
pipenv itself is a third-party package, so the first step is to install it.
It is okay to have this in your
site-packages directory, as you’re always going to want it. Once this is in place, it will be virtual envs from here on in. Let me just scroll up a bit.
02:55 Nothing here you haven’t seen before. Package is installed, along with all of its dependencies.
I’m going to create a new project called
To create a new virtual environment, you use the
pipenv shell command.
shell command is used for entering a virtual environment, but if it doesn’t find one, it creates it, like in this case. There are a few things going on here. First off, it tells you that it is creating a virtual environment and a Pipfile.
03:31 The Pipfile is one of two files used to specify your packages. I’ll get into details with those in a second. Next, it tells you what version of Python it’s going to use. In my case, it was C-Python 3.10.
It also adds some useful base packages to the virtual environment—those for doing package management,
pip, setuptools, and wheel. Four lines from the bottom, it tells you where the virtual env files will be stored.
pipenv uses a centrally located place for all of its files. Your location is OS specific. Mine is my home directory
.local/share/virtualenvs. Note that it names the virtual env with the name of my directory and a hash.
This naming is how it knows which environment to activate on subsequent calls to the
shell command. Finally,
pipenv activates my virtual environment.
04:19 You can see that it modifies the prompt to tell me what environment and directory I am in. Depending on your settings, this may also include your host and username information.
The virtual environment is created in a central location, but creating it creates a local file, Pipfile. This, combined with another file called
Pipfile.lock, is the replacement for
04:43 These files specify information about the packages you’re using in your project. Let’s take a look at one.
04:52 Here’s the default Pipfile before you install anything. It uses the TOML format, which is similar to Windows INI files. Sections in the file are denoted by double square brackets, and the sections themselves contain key/value pairs.
The source section specifies where to get packages from. This defaults to the PyPI repository. The
dev-packages section specifies packages that are installed. Right now, these are empty, and the
requires section specifies the version of Python being used.
05:25 Now that you’ve seen the file, let’s install something and look at the difference. Let’s repeat what was done in the previous lesson, but this time in a virtual environment.
The command is similar, but this time it is
pipenv install. The output is a bit different, but the result is the same. The package is installed.
pipenv doesn’t bother telling you about what
requests depends on. It just updates the files. Looking in the directory, you’ve now got both a
Pipfile and a
Let’s look inside of them. Here’s the state of the Pipfile after I did the installation. The only change here is to the
packages section. It has an entry for the
The asterisk there indicates that any version of
requests is allowed—i.e., it should grab the most recent. It did that because I didn’t ask for a specific version when I did the installation. After you’ve done the installation, you can always edit the Pipfile to change the behavior, so if you wanted to pin
requests, you could do that by editing this file.
06:36 That being said, you don’t normally do that because the other file, the lock file, is for pinning.
Pipfile specifies what packages you install.
Pipfile.lock contains a fully pinned listing of all the dependencies. If you want a production environment that exactly mimics your dev environment, you can do an installation with the lock file instead. On the other hand, as you’re developing, if all you care about is the libraries you installed, you can continue to use the Pipfile itself and then create a new lock when you’re ready to go. The format of the lock file is JSON.
07:14 It is made up of a series of nested dictionaries. The first part here specifies some metainformation and indicates what version of the Pipfile spec is being used and which Python version. The third section states where the libraries were sourced from, in this case the PyPI repository.
The next section of the lock file contains an entry for each of the libraries installed in the virtual environment. Each library is named, has a hash describing its contents, and fully pins a version. On the screen right now are the pinned versions of
Wow, lots of screenfuls of lock files. This is the last one. The
default section continues on this screen here, pinning the
Finally, at the bottom, there’s a section for libraries only to be used in the dev environment, which at the moment is empty. Let’s play around with
pipenv some more.
Before showing you the contents of
Pipfile.lock, I was inside of the
hola virtual environment. Back in the terminal now, I use the
exit command to get out of that shell.
08:28 Let’s create another project.
pipenv shell inside of the new
salve directory creates a new virtual environment and activates it. Just like
pipenv install supports the
hello package? Let’s try it again.
So far, so good. And now the
Well, that didn’t go well. Unlike
pipenv has a very strong opinion about conflicts. It won’t let you do it by default. Let me scroll up just a bit here.
pipenv attempted to create the lock file, and the locking failed because of the conflict. And it explicitly tells you that. About halfway down here, you’ll see the keyword
critical, and it tells you exactly what conflict happened:
0.4.4 were specified by two different libraries. If I scroll back down, you’ll see it even gives you some suggestions as to what you can do about it.
Now, if you want to shoot yourself in the foot, you’re going to have to do it explicitly.
pipenv puts the safety on the gun and gives you steel-toe boots.
10:03 You can still pull the trigger, but it shouldn’t happen by accident.
You’ve seen the basics, but wait, there’s more! It slices, it dices, it even juliennes! Next up, more
Become a Member to join the conversation.