Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Python Packages
In this lesson, you’ll learn about Python packages. Suppose you have developed a very large application that includes many modules. As the number of modules grows, it becomes difficult to keep track of them all if they’re dumped in one location. This is particularly true if they have similar names or functionality. You might want a way to group and organize them.
Packages allow for a hierarchical structuring of the module namespace using dot notation. In the same way that modules help avoid collisions between global variable names, packages help avoid collisions between module names.
Creating a package is quite straightforward, since it makes use of the operating system’s inherent hierarchical file structure. Consider the following arrangement:
Here, there is a directory named pkg that contains two modules, mod1.py and mod2.py. Take a look at the contents of the modules. Here’s mod1.py:
def load_data():
print('loading data using mod1.load_data()')
class Customer:
pass
Here’s mod2.py:
def clean_data():
print('cleaning data using mod2.clean_data()')
class Location:
pass
Given this structure, if the pkg directory resides in a location where it can be found (in one of the directories contained in sys.path), you can refer to the two modules with dot notation (pkg.mod1, pkg.mod2) and import them with the syntax you’re already familiar with:
import <module_name>[, <module_name> ...]
Here’s what that looks like in practice:
>>> import pkg.mod1, pkg.mod2
>>> pkg.mod1.load_data()
loading data using mod1.load_data()
>>> x = pkg.mod2.Location()
>>> x
<pkg.mod2.Location object at 0x10818c550>
You can also use this syntax:
from <module_name> import <name(s)>
Here’s what that looks like in practice:
>>> from pkg.mod1 import load_data
>>> load_data()
loading data using mod1.load_data()
You can also use this syntax:
from <module_name> import <name> as <alt_name>
Here’s what that looks like in practice:
>>> from pkg.mod2 import Location as Primary
>>> y = Primary()
>>> y
<pkg.mod2.Location object at 0x10d8ed2b0>
You can import modules with these statements as well:
from <package_name> import <modules_name>[, <module_name> ...]
from <package_name> import <module_name> as <alt_name>
Here’s what that looks like in practice:
>>> from pkg import mod1
>>> mod1.load_data
<function load_data at 0x101b37f28>
>>> from pkg import mod2 as foo
>>> foo.clean_data()
cleaning data using mod2.clean_data()
You can technically import the package as well:
>>> import pkg
>>> pkg
<module 'pkg' (namespace)>
But doing that is of little use. Though this is, strictly speaking, a syntactically correct Python statement, it doesn’t do much of anything useful. In particular, it doesn’t place any of the modules in pkg into the local namespace:
>>> pkg.mod1
Traceback (most recent call last):
File "<input>", line 1, in <module>
pkg.mod1
AttributeError: module 'pkg' has no attribute 'mod1'
>>> pkg.mod2.Location
Traceback (most recent call last):
File "<input>", line 1, in <module>
pkg.mod2.Location
AttributeError: module 'pkg' has no attribute 'mod2'
>>> pkg.mod1.load_data()
Traceback (most recent call last):
File "<input>", line 1, in <module>
pkg.mod1.load_data()
AttributeError: module 'pkg' has no attribute 'mod1'
To actually import the modules or their contents, you need to use one of the forms you saw above.
00:00 Up to now, you’ve only had a single module to import. But what happens as you start to build a much larger application? Well, that’s where Python packages can come in.
00:10 When you’re trying to keep track of that growing number of modules, creating a package is typically the solution. Packages allow for a hierarchical structuring of the module namespace, again using dot notation.
00:23
So, here’s an example package structure, and you’ll get to play with this in a moment. At the top level, you again have a directory, and let’s say that directory is named pkg (package), and inside of it, it includes mod1.py, and then it has a second module, mod2.py.
00:40
So now, both of these modules are inside of the directory hierarchy of pkg/. Well, how do you import them and work with those modules? Let me have you practice.
00:51
Inside your directory ModulesAndPackages/, I want you to create a new directory pkg.
00:59
So here’s the directory pkg/ and inside of it, you’re going to create two modules, mod1
01:16
is going to have a function called load_data(). load_data() when called is going to print out a line of text about the module.
01:38
It also has a class. This class is going to be the Customer class. Okay, so there’s a function named load_data() that’s part of mod1, and then there’s a class called Customer. Again, just examples.
01:49
mod2 is going to have a function named clean_data(). And again, right now it’s just going to be really simple.
02:08
mod2 will have a class also. This class will be the Location class.
02:15
So mod1—and you can see it here with load_data() and Customer. mod2 with clean_data() and Location. Great.
02:22
Now, you’re still in the root directory of ModulesAndPackages/, so if you were to start your REPL session,
02:32
what are the ways that you would import packages or modules from packages? Well, you can use dot notation. In fact, you could import both of them at once, so import pkg.mod1, pkg.mod2.
02:48
And what you’ll see here is that under pkg, both of the modules are available and under mod1, there’s the load_data() function, and you could see it running there. And you could create an object, say x = pkg.mod2, and x is going to be a Location. Marking the spot, if you will. Great!
03:10
So x is a Location object from mod2 from the package pkg. Nice. So, that’s one way, is just to import them.
03:20
Now you’ve learned multiple ways to import. Another way would be to just import individual items, so from pkg.mod1, you could import just the function, load_data.
03:35
So now inside of here, load_data() is in the local symbol table and is available, and you can see it running. So again, you could import from the package individual items from the individual modules using dot notation. Or you can also use the alternative name technique.
03:53
Let’s say you want to import that class Location, but instead of calling it Location, you want to call it Primary.
04:01
So now you could create a new object using that class, so now y is a Location object—again, it’s using the name that you gave it of Primary. Go ahead and exit, and re-enter the REPL.
04:18
Another way that you learned to import before was using the from statement. So from pkg you could import mod1. So, instead of the dot notation of having pkg first, you actually would then just use mod1. And similarly, if you wanted to, you could load from the pkg, import mod2, and then give it a different name, let’s say foo.
04:44
So now foo has clean_data(), and if you wanted to, you could call that function. It’s good to practice all these techniques again.
04:54
The last thing that you might be thinking is, “Can you import everything typing in import pkg?” Well, that imported pkg, but not a lot happened.
05:05
pkg is a module and it shows that it has a namespace, but is mod1 available? No. Is pkg.mod2.Location? No. Could you call any of the functions? Nope.
05:23 So what this shows is that you have to use any of those previous styles of importing. You can’t just import the package by itself—at least not yet. There may still be a way.
05:37 In the next video, you’re going to dive into package initialization.
Ricky White RP Team on July 5, 2020
Try adding a __init__.py file to your pkg folder. There doesn’t have to be any code in it, but it’s presence will help it be identified as a package that can be imported.
macblanelw on July 6, 2020
This does not work for me: bpython version 0.18 on top of Python 3.8.2 /usr/bin/python3
import pkg.mod1, pkg.mod2 Traceback (most recent call last): File “<input>”, line 1, in <module> import pkg.mod1, pkg.mod2 ModuleNotFoundError: No module named ‘pkg’
Clearly, my version of python does not assume that a directory is a module, or will not treat the directory as one. Anything I can do? My current path is ~/projects/python/ModulesAndPackages/pkg/ I tried this from within /pkg/ and above it. Same results.
Chris Bailey RP Team on July 7, 2020
Hi @macblanelw, Are you using the resources for the course? I just downloaded the resources from “Supporting Materials” button, and unzipped the code to my desktop on my Mac. I entered into my terminal and changed into the sample directory, cd 08_PythonPackages.
The directory looks like this when I use the ls command:
pkg
I then started bPython
$ bpython
bpython version 0.19 on top of Python 3.8.2
I was able to confirm it was working when I typed import from the prompt and the letter pk, as bpython showed auto completion options for:
>>> import pk
pkg pkg_resources pkginfo
Have you been able to use the other resource files? And have you been able to create and use some of your own in a similar way? This is
konkna on July 16, 2020
hey,
when I have downloaded the code and run it via command line it works.
/mnt/c/NK/Learnings/python/realpython/ModulesAndPackages_Code/ModulesAndPackages_Code/08_PythonPackages
$ python3 08_PythonPackages_REPL.py
loading data using mod1.load_data()
loading data using mod1.load_data()
cleaning data using mod2.clean_data()
loading data using mod1.load_data()
However, when I run it via the bpython3 repl it doesn’t unless I have the init.py file
and even setting the sys.path.append to the directory doesn’t work.
>>> import sys
>>> sys.path
['', '/usr/bin', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dy
nload', '/home/konkNa/.local/lib/python3.6/site-packages', '/usr/local/lib/python3.6/dist-
packages', '/usr/lib/python3/dist-packages']
>>> sys.path.append(r'/mnt/c/NK/Learnings/python/realpython/ModulesAndPackages_Code/Module
sAndPackages_Code/08_PythonPackages')
>>> import pkg.mod1
Traceback (most recent call last):
File "<input>", line 1, in <module>
import pkg.mod1
ModuleNotFoundError: No module named 'pkg'
Wondering how would be it different running in the batch/script mode vs running in the repl
FYI.. I use a ubuntu on the windows machine. So, it has python2 which is default and python3.6 and python3.7
bpython3 points to
bpython3
bpython version 0.17.1 on top of Python 3.6.9 /usr/bin/python3
tirexxerit on July 17, 2020
I got the same bpython error when I use python3.6 but if I switch to python3.8, I don’t see any error. Look;
bpython
bpython version 0.19 on top of Python 3.8.4 /home/gyilmaz/Automation/Python/PAN/Dev/Training/venv/bin/python
>>> import pkg
>>>
bpython3
bpython version 0.17.1 on top of Python 3.6.10 /usr/bin/python3
>>> import pkg
Traceback (most recent call last):
File "<input>", line 1, in <module>
import pkg
ModuleNotFoundError: No module named 'pkg'
tirexxerit on July 17, 2020
or it is about bpython version not necessarily about python as it works on 0.19 bpython..
Idris Diba on Aug. 19, 2020
I also got the same error but then I added __init__.py to pkg folder which is located inside Modules_and_packages and it worked like a charm.
Dan B on Nov. 17, 2020
I also got the same error, but then I added __init__.py to pkg folder, which is located inside Modules_and_packages, and it worked like a charm.
Become a Member to join the conversation.
macblanelw on July 4, 2020
This does not work for me: bpython version 0.18 on top of Python 3.8.2 /usr/bin/python3
Clearly, my version of python does not assume that a directory is a module, or will not treat the directory as one. Anything I can do? My current path is ~/projects/python/ModulesAndPackages/pkg/ I tried this from within /pkg/ and above it. Same results.