Installing Packages Using pip

00:00 In the previous lesson, I gave an overview of the course. In this lesson, I’ll show you Python’s built-in package installation tool, pip. Python has a long history of package management, and there have been several different tools over the years.

00:14 This can be quite confusing sometimes, especially if you’re working on older code or packages. It is also an area that is in flux in the language. People are still trying to solve certain thorny package management problems.

00:27 The most up-to-date of the package management tools that ships with Python is called pip. It is a command-line tool that installs third-party packages from the PyPI package repository. You can install packages from elsewhere as well, but that’s the default. When you install a package using pip, the package goes in the site-packages directory of your interpreter’s installation. This is problematic if you’re managing multiple projects that require different versions of the same package. Virtual environments are a solution to this problem, but for now, let me show you how pip works without the virtual env.

01:04 Python has a list of places it looks when you import code. This list can be seen in the sys-module’s path value. The code on the screen here prints the contents of this list to the terminal.

01:16 Let me run it in a shell.

01:23 For this demonstration, I’m using a clean installation of Python 3.10. I’ve done this so that you can see the base state of things before I start installing packages. The results of the show_path script print out the various places Python will look for code.

01:38 The first item is my current directory. The rest of them are deep within the installation of the interpreter. I’m running this code on the Mac. The /Library/Frameworks place is where Mac keeps all this stuff.

01:51 If you’re using a different kind of operating system, your values will be different. No matter what OS you’re using, there will be several directories here where Python looks for code.

02:00 The last one, site-packages, is where Python puts any third-party libraries that that you may install.

02:13 Looking inside the site-packages directory, you’ll see a few things. The two key ones are pip and setuptools. Both of these are package installation tools.

02:23 Python puts its packaging tools in the same place it will put anything you install. The pip command has a bunch of subcommands that allow you to do things.

02:35 I’m using a Python 3.10 version, hence the full name pip3.10. You may just need pip depending on where your installation is and how you’re running things.

02:45 The freeze subcommand lists all the current packages that are installed. Nothing was returned here because nothing yet has been installed. It’s a clean base system. All right, then, so let’s install a package.

03:03 You install packages by using the pip install subcommand, passing in the name of the package you want installed. In this case, I’m installing colorama.

03:12 It’s a popular library that makes it easier to use colored text in a terminal. Using the pip install command outputs some information to the screen.

03:21 It tells you that it found the package. In my case, it found it in my local cache. That’s because I’ve installed it before. If I hadn’t, it would go off to PyPI by default to get it.

03:33 It also tells you exactly which form of the package is installed. In this case, it was version 0.4.4, which is both Python 2 and Python 3 compatible using the wheel format.

03:45 The wheel format is one of several different ways of assembling a package. You can think of it kind of like a ZIP file containing the contents of the code to be installed. It goes much deeper than that as it also could contain code that can be compiled locally, metainformation, and different content depending on your version of Python.

04:04 But for now, just know that wheel contains the third-party library being installed. With colorama installed, let’s run freeze again.

04:15 This time, there’s something installed. pip tells you it found the colorama library, version 0.4.4.

04:29 Looking inside the site-packages directory again, you can see two new things, the colorama directory and the colorama-0.4.4.dist-info directory.

04:39 These two folders contain the code and metainformation about the install, respectively. There really isn’t any magic here. The colorama folder contains code, just like code you might write in your own directory. In fact, you can edit the files in colorama, and it will affect what happens when you import the library. You want to be careful if you’re doing this, of course, as you’re making changes from the original, and they only will be local to your installation, but I dig around in the site-packages directory all the time when I’m coding.

05:07 If you want to better understand what someone’s library is doing, you can look at the code directly. There are some caveats here. If the library uses C extensions, then the directory will contain compiled code, but if the library is Python based, you can see it in all its gory details.

05:23 When I ran pip install, I gave only the name of the library. This will get you the latest version. You can also ask for a specific version.

05:37 Specifying a specific version is done using the same format as the output from pip freeze. Here, I’ve asked it to install colorama-0.3.3.

05:48 You can also get fancy with your installation and ask for packages in a range. For example, you could say you want something greater than 0.3.3, but less 0.4.0. In that case, you’d get the largest dot-3 release, which for colorama would be 0.3.9.

06:06 Notice a few things in the output here. For starters, because version 0.3.3 is older, it didn’t have a wheel package. Not a problem. pip knows how to use the older formats as well and tells you so. Second, colorama-0.4.4 was already previously installed. Because of this, pip uninstalls it first, then replaces it with 0.3.3.

06:29 Spoiler alert: this feature could cause you some heartache, but I’ll get to that in a couple of minutes. Running freeze again, and you see that 0.4.4 is gone, but 0.3.3 is there.

06:47 You can also use pip to uninstall a package using the uninstall subcommand.

06:57 I’ve used the -y switch to answer yes to the are you sure message that happens per package by default. pip found 0.3.3 of colorama and uninstalls it. I’m back to having a clean base with no packages. colorama is a small library with no dependencies.

07:16 Let’s install something that’s a bit more complicated.

07:23 requests is a fantastic library for grabbing content from the Web. The library uses other libraries, which pip fetches automatically.

07:31 Some of these were in my package cash, and some came from the Internet. Remember when I mentioned that you can specify ranges of package versions? You can see some examples of that here.

07:41 The certifi dependency must be more recent than version 2017.4.17, and the urllib3 dependency must be between 1.21.1 and 1.27.

07:55 You can use these same formats when you’re calling pip install if you want to restrict your package ranges the same way. The last line of output from pip tells you everything that happened as a result of the install requests command. In this case, five different packages got installed. Back to pip freeze, and there are the five installed packages.

08:28 And it shouldn’t be a surprise by now that inside of the site-packages directory, you can see all the newly installed stuff. In order to get back to a clean base, you have to specifically uninstall each of the packages.

08:41 If you just uninstall requests, it doesn’t take its dependencies with it.

08:53 Once again, I’ve used the -y this time to avoid being asked are you sure five times, once for each package. With the uninstall complete, I can run pip freeze again.

09:07 Nothing comes back. So I got them all. By default, pip gets packages from the PyPI repository on the Internet. You can also install packages from your local machine.

09:18 This can be very handy if you’re creating and testing your own packages. Inside the sample code, you’ll find two folders, hello and bonjour, both of which are installable packages. To keep things clean, I’m going to create a directory

09:37 and then go inside it.

09:41 This isn’t strictly necessary, but it will be helpful when I start playing with pipenv in the next lesson.

09:51 Using the -e parameter with pip install does an editable install. This is an install from the local file system. It is called editable because site-packages gets a link rather than a copy of the package, meaning you can still edit the package in the original directory.

10:08 This is very useful if you’re debugging your own packages before shipping them. The hello package includes a dependency on colorama-0.3.9.

10:18 Like with requests, pip sees the dependency and installs it along with hello.

10:28 Doing a freeze shows the colorama library and the hello package, and a comment telling you that the hello library was installed in editable mode.

10:37 If hello had version control information in it, like a .git folder, this would also show you what the upstream repo’s URL was.

10:46 Let me quickly show you how hello specifies its dependencies.

10:54 Inside the hello directory, there is a file called setup.py. This is a script that tells the packaging tools about the package.

11:02 There are actually several different ways of doing this, and the ins and outs of building packages is rather complicated. The script takes a dictionary containing attributes about the package, one of which is called install_requires. This attribute takes a list of the names of packages that hello is dependent on. In this case, it is colorama 0.3.9. Like with pip install, you can specify a range of packages by using other comparison operators than the == (double equals), like here. For the purposes of this course, it’s not important to understand the details of how the dependencies are specified inside of a package.

11:37 Just understand that there’s a list there, and it lists those dependencies. All right, so hello was installed. Let’s install its friend, bonjour.

11:53 Well, that was problematic. Like hello, bonjour also depends on colorama, but this time on a different version number. pip discovers this and conveniently uninstalls the older version for you. When I say “conveniently,” I mean it in the sense of “conveniently, I pointed the gun at my foot.” pip does its best to warn you about the potential of losing toes, but it is just a warning. Why is this a problem? Well, it all depends on the library.

12:20 The good folks who wrote colorama have kept their code fairly backward-compatible. In all likelihood, hello will still work with colorama 0.4.4., even though it says it is dependent on 0.3.9, but there’s no guarantee in this case.

12:37 By “conveniently” upgrading the dependency, it is possible that hello is now broken. To make matters worse, this could have been the other way around as well.

12:46 Had I installed bonjour first, the dependency would have been downgraded, which is a recipe for disaster. There’s probably a reason the writer of bonjour pinned colorama to the 0.4.4 version.

12:57 It might not work with something earlier. This is why virtual environments are considered the best practice. The more projects you have on your system, the more likely it is that you’re going to have a dependency conflict.

13:08 Remember it isn’t just your projects’ dependencies, but also the dependencies of all the libraries that you depend on.

13:19 Running pip freeze shows that 0.3.9 was replaced with 0.4.4, and the two editable installs are there. Now I’m going to run uninstall and clean everything up.

13:36 Given the potential problems with library dependencies, a common practice is to pin your versions. This means recording exactly those libraries your project needs and making sure that if you deploy somewhere else, you get the same thing.

13:49 To see this in action, let’s install requests again.

13:59 Running freeze shows exactly what libraries are installed. You can redirect the output of this and store that information to pin the values.

14:14 The most common name for a file that you pin your versions in is requirements.txt. To see the value this provides, let’s uninstall requests.

14:27 Remember, this didn’t get rid of the dependencies, just the requests library itself. So right now there are four packages instead of five. You can use the -r argument to pip install to install a list of packages from a requirements file.

14:46 In this case, most of the packages were already satisfied because I only uninstalled requests. pip install sees this and installs the missing requests package.

14:59 And you’re back in business. Before pipenv, requirements.txt was the most common way of pinning the dependencies in a project. pipenv does this differently.

15:11 I’ll cover that more in the next lesson. You’ve seen how pip works. You’re almost ready to move on to pipenv, but first, like all good little children, you need to clean up your toys. Let’s uninstall everything so that I’m back to a clean base before the next lesson.

15:37 Phew! That was a lot of package management. Next up, let’s do some pipenv.