Virtual Environments

00:00 Hey folks, how’s it going? This is Dan with another Python trick. Today I’m going to talk about Python virtual environments. So, Python actually includes a really powerful packaging system to manage the dependencies of your program, so other modules that your program would use that are not part of the standard library and that you didn’t write yourself.

00:20 And you probably already used that to install packages or third-party packages through the pip command. One really confusing aspect about installing packages with the pip command is that by default it tries to install them into your global environment.

00:37 The good thing is that this makes any new packages available globally on your system, which is great for convenience, but it also turns into a nightmare if you’re working with two different projects—or more—that actually require different versions of the same package, right?

00:51 So, let’s say you have one app that you’re building that requires Django 1.10.0, and then the other one requires Django 1.10.8. And if you only have a single global environment for your Python packages, then that’s not going to work—you’re going to run into version conflicts. And now, even worse, the same thing could happen if you’re working on some programs that require Python 2, and some that require Python 3.

01:12 Where would you install this stuff? Now, by default Python 2 and 3 actually use different environments, but what if you were working on a program that actually was running on Python 3.3 and then now you were starting a new program and it would require Python 3.6?

01:26 How would you actually install those in parallel? So, the solution to all of these problems is so-called virtual environments, because they allow you to keep all of your dependencies separated by project, including different versions of the Python interpreter.

01:40 So, a virtual environment is an isolated Python environment and it physically just lives inside a folder that contains all of the packages and other dependencies, like native code libraries that a Python project would need.

01:52 Now, to show you how these virtual environments work, or also virtual envs is how they’re called for short, I’m going to give you a quick example of that, and then we’re going to install a third-party package into it. So, first of all, we’re going to take a look at where the global environment currently lives, and we can do that with the which command. In this case, I’m going to be using Python 3, which means that I’m going to add this 3 suffix to all of my commands here, like pip3, python3, and so on. All right, so the first thing that we’re going to do is we’re going to take a look at where the global environment currently lives, right?

02:28 And I’m going to do that with the which command, which tells me the path to a file or to a executable. And in this case, I’m looking at the pip3 command because I’m using Python 3 here.

02:39 So if I run this, it tells me that my Python install lives inside /usr/local/bin/—or, my pip install lives inside /usr/local/bin/. This is typical if you’ve installed that through homebrew on macOS, for example.

02:52 Now, I’m going to pretend I’ll be working on a new Python project here, so I’m going to create a new directory for this project, and then I’m going to change into this project directory.

03:04 And what I’m going to do now is I’m going to create a virtual environment inside this folder here. So, what I’m going to do here, I’m going to use the magic incantation python3 -m venv, which stands for virtual environment, and I’m just going to tell it to create a virtual environment inside this my-python-project/ folder, and I want it to create that inside a new subdirectory called venv, which is just a naming convention that I like to use.

03:31 And this is going to set up a new virtual environment. This should finish any minute now. All right, so we just created the new virtual environment, and now in a look at the contents of my project folder, previously it was empty and now I’ve got this venv/ folder in there, and then when we look inside it, you can see there’s a bunch more directories and pretty much, like, a full Python install. And we can drill down further just to show you that there’s a bunch of stuff in this folder here!

04:00 So, when we just list a directory tree here, you can see that we’ve got this bin/ folder, this include/ folder, the library lib/ folder.

04:09 This is where all of our Python third-party packages are going to live, and they were seeded with some basic stuff, like a very basic Python install. And that’s what a virtual environment does. Now we’ve pretty much got this tiny self-contained virtual environment—a Python environment—installed inside our project folder. Now, if we run the which pip3 command, you can still see that while we set up the virtual environment, running pip3 would still execute inside the global environment, so what you need to do here is you need to load a script inside the venv/ folder—and again, this is just by convention—where it’s called ./venv/bin/activate.

04:49 You want to run that inside your shell. On Windows, that would just be a little executable, so you wouldn’t use the source, you would just kind of run this activate command. And what you can see now, it actually changed my prompt here, because now this particular venv is activated and it tells me so by adding this little notice here to my prompt, right?

05:07 And so if I go ahead and run which pip3 again, you can see now that it actually points to a different path inside the virtual environment. And this is great because that means now when I install stuff, it will go inside the virtual environment rather than the global environment, right? So, I can also use the pip3 list command and that’s going to show me what’s installed here. And you can see here—this is just very basic stuff, right?

05:33 So this is a very basic Python install with the standard library and then pip and setuptools, which—again, are now part of the standard library.

05:41 So, this is a brand new Python environment. And now I can go ahead and say, “Let’s install a module here,”

05:49 and this is going to download and install this package. Now when I do a list, then you can see here that we just installed schedule inside here.

05:59 Actually, from this point on once I’m inside the virtual environment, I can just go and use pip instead of pip3, because what enabling or activating a virtual environment will do is it will actually link up the pip just to point to the right version of pip, and the same is true with Python, right? So if I go python, I’m going to run Python 3.6 out of this virtual environment.

06:23 So, what I’m going to do here, just to show you that we actually installed this module correctly, I’m just going to import the schedule library and then just do a dir() on that. And then you can see here, we’re able to install this package and we were able to sort of inspect it and made sure that it was imported correctly, which is great because this is really what we wanted here. All right. So now, you know what a virtual environment is, how you create one, how you activate one. Now the question is, “Okay, how do I get back to the global environment?” And it’s pretty easy to do. So, when you’re inside a virtual environment you can just use the deactivate command, which is part of any enabled, any active virtual env.

07:03 So when I run this command, it’s going to take me back to the global environment, and now when I go which pip3 it’s going to point to the global environment here again.

07:13 And then I could also do… if I wanted to go back into the virtual environment, I would do the same thing and it would turn it back on. All right, now I’m back in, and now we’re back out.

07:28 Cool! So again, you know, a very simple tool actually, once you kind of understand how to use it and how the basic use cases for it work, but these virtual environments are super helpful because they help you keep your project dependencies in separate places, they’re good for security—I mean, it’s obviously not perfect because you’re still running a bunch of code under your local username, but it’s better than running it as root when you install packages—and they help you avoid version conflicts. And as a best practice, in general, all of your Python projects should use virtual environments to store their dependencies so that you can avoid conflicting versions and all of these headaches.