Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Getting Started With pip and PyPI
Python’s package manager is called pip, and it comes bundled with every recent version of Python. pip allows us to install packages that don’t come bundled with the Python standard library.
By default, pip searches what’s called PyPI, or the Python Package Index. This public repository contains thousands of packages written by the Python community.
This lesson shows you how to use pip to download the requests package from PyPI. This package allows you to conveniently send HTTP requests with Python:
import requests
url = 'https://www.google.com'
response = requests.get(url)
print(f'Response returned: {response.status_code}, {response.reason}')
print(response.text[:200])
Packages can be installed with pip install. To view a full listing of commands, run pip help.
pip list will show you a list of all the currently installed packages. pip show will show you more information about a package, including packages it requires (dependencies) as well as packages that require it (packages it is a dependency of).
The following command string can be used with conda to create a new, minimal virtual environment called web-parser:
$ conda create --name web-parser python --no-default-packages
You can also specify a custom python version:
$ conda create --name web-parser python=3.6 --no-default-packages
To ensure that you only pip install safe, effective packages, check out How to Evaluate the Quality of Python Packages.
00:00
Python’s package manager is called pip, and it comes bundled with every recent version of Python. You can also add it to an existing Python installation with a tool called ensurepip.
00:14
pip allows us to install packages that don’t come bundled with the Python standard library. When you use pip to install a package, it first attempts to locate the package that you desire. By default, it searches what’s called PyPI, or the Python Package Index. PyPI is a massive collection of packages developed and maintained by the Python community.
00:41
You can use pypi.org to browse for new packages. One of my favorites is called matplotlib, so I’ll search for that package. On the left, you can see we can Filter by classifier, such as showing only stable versions for production, or beta if we want to try new features of our favorite packages.
01:06
Here’s the version I’m looking for. You’ll see at the top that underneath the name of the package, PyPI tells us the required pip command to install it.
01:17 We also get a description of the package in the middle and some links relevant to this package on the left. We can even see the Release history.
01:30
Before we can write any programs, we need to create a new virtual environment to store Python, pip, and all of our packages. If you’re using Conda, the command to create this new environment is simply conda create. For the --name, I’ll give it web-parser. For the Python version, I’ll say 3.6. There’s no particular reason for this version, but I just wanted to demonstrate that you can install a specific version of Python if you’d like.
02:05
I’ll also add this --no-default-packages flag, which will greatly reduce the amount of packages Conda installs into our new virtual environment.
02:17
It’ll prompt you to confirm, so press y and hit Enter.
02:25
Once that’s installed, we can use the new virtual environment with conda activate web-parser.
02:34
This will ensure that our terminal uses the versions of Python and pip local to this virtual environment, instead of the global versions installed on our system.
02:46
You can tell it’s activated because the name of the virtual environment shows before the prompt. To demonstrate our new isolated Python development environment, I’ll type which python.
03:00
If you’re on Windows, replace which with where. The path in the output tells us that our Python programs will run using the Python interpreter within the web-parser environment.
03:14
We can do the same thing for pip.
03:19
To see a list of all the possible program arguments you can pass to pip, run pip help. And if I scroll up here, you’ll see that there is a lot we can use pip for. The package I’ll be using in my program is called requests, which allows us to send HTTP requests just like a web browser would. To install it, run pip install requests.
03:48
Here, requests is called the requirement specifier. We can type in just the package name to download the latest stable version, or manually specify a version by appending a dash (-) and then the version number after the package name.
04:07
Once I press Enter, pip will show us all of the packages it’s collecting, as well as where it’s downloading them from. These are all the dependencies needed for requests.
04:19
It looks at the package metadata to know what the package’s dependencies are, and then it installs them. If you get a message saying that pip is out of date, you can upgrade it with this command: python -m pip install --upgrade pip.
04:41
Now that we’ve installed requests, we can list all of the packages we have installed with pip list. If all has gone smoothly, you should now see requests in this list.
04:57
The rest of these packages, Conda installed automatically with our new virtual environment. To show more information about any package, use pip show.
05:11
Notice the last two lines: Requires and Required-by. Requires shows us the packages that this package depends on. Required-by shows us what other packages we have installed that depend on requests—and it looks like we don’t have any.
05:31
Since we’ve confirmed that the requests package has been installed, we’re now ready to write a program that actually utilizes it. Normally, I would use Visual Studio Code, but I’m going to do something a little bit more fun.
05:45
I’ll use a version of the infamous VIM editor, built in Python. It’s called pyVim, and it too can be acquired with pip.
06:00
To create a new file and open it in pyVim, type pyvim and then a filename, like using-requests.py. Once pyVim is open, press i to go into insert mode, and now we can start typing. Of course, we’ll first have to import requests.
06:24
Now, we can define a url variable and grab its response to a GET request, just like this. This next line of code will print the response status code and the reason.
06:39 And also, the first 200 characters of the HTML that’s returned to us.
06:45
Press Escape to exit insert mode and :wq Enter to save and quit pyVim. To run this script with the Python interpreter, we can do python using-requests.py.
07:05
And there’s our output! It looks like everything worked—well—200, OK.
Bartosz Zaczyński RP Team on Sept. 21, 2020
It should be. What platform are you on? What version of Python did you install? Are you using plain CPython or some flavor like PyPy or distribution like Anaconda?
petrovicdjordje750 on Dec. 29, 2020
For Windows pythonistas:
Open PowerShell and navigate to folder where you want to create virtual environment.
For creating virtual environment run:
python -m venv web-parser
To activate virtual environment run:
.\web-parser\Scripts\activate or .\web-parser\Scripts\Activate.ps1
To deactivate, just type deactivate, this will call a global function defined in Activate.ps1 script, which will deactivate virtual environment.
To get python interpreter path run:
where.exe python
Notes:
- When creating virtual environment u can chose version by replacing
pythonwith specific version (e.g.python3.8.exe) - “web-parser” is just a name and path of virtual environment folder, in this case “web-parser” folder will be created in the current folder.
- I am not sure how to pass
--no-default--packagesflag upon creating virtual environment, for more information you can refer to python documentation
amirrastkhadiv on Dec. 30, 2020
Hi, could you please tell me what is the difference between following methods to upgrade pip? When we must use each of them?
$ python -m pip install --upgrade pip setuptools$ sudo pip3 install --upgrade pip setuptools
amirrastkhadiv on Dec. 30, 2020
Could you please provide further information about the ensurepip?
Bartosz Zaczyński RP Team on Dec. 30, 2020
@amirrastkhadiv The sudo command executes the following command as the superuser (root).
Specifying pip3 will search for it in the directories listed in the $PATH variable. Therefore, it’ll use the pip command of the globally installed Python 3 interpreter.
On the other hand, when you specify python -m pip, it’ll run the pip module using whatever Python interpreter the python command is currently pointing to. You can check that by issuing which python in your terminal:
$ which python
/home/jdoe/.virtualenvs/foobar/bin/python
It may point to an active virtual environment, for example.
roberts on Oct. 30, 2021
I’ve just spent 3 hours of my life which I’ll never get back, with Python 3 there is no pip but pip3!
Bartosz Zaczyński RP Team on Nov. 2, 2021
@roberts Oh, man. I’m sorry you’re feeling frustrated. Everyone has been there many times, so don’t get discouraged. It’s an unfortunate side-effect of learning new concepts.
Whether the pip command is available or not depends on how you installed Python and what your operating system is. However, you’re not supposed to call pip3 directly because that invokes the system-level interpreter.
In most cases, you’ll want to isolate your projects and install their dependencies into sandboxed virtual environments. You can do so by typing these commands:
$ python3 -m venv venv/
$ source venv/bin/activate
(venv) $ python -m pip install requests
The first one creates a copy of your Python interpreter in a local folder named “venv/”. The second one activates it. Notice that activating a virtual environment rewires the python and pip commands, so you no longer have to type python3 and pip3.
Finally, it’s considered the best practice to call pip as a Python module with the python -m pip command. It will reduce the risk of running pip in a different environment or some shell alias, which might be pointing to the wrong virtual environment.
Become a Member to join the conversation.
kethan on Sept. 19, 2020
pipcommand is not installed in Python