Subpackages

Python Modules and Packages: An Introduction Christopher Bailey 06:04

In this lesson, you’ll learn about subpackages. Packages can contain nested subpackages to arbitrary depth. For starters, let’s make one more modification to the example package directory as follows:

The four modules (mod1.py, mod2.py, mod3.py, and mod4.py) are defined as they were before. But now, instead of being lumped together into the pkg directory, they are split out into two subpackage directories: sub_pkg1 and sub_pkg2.

Importing still works the same as you saw before. Syntax is similar, but additional dot notation is used to separate the package name from the subpackage name:

Python
      
        
      
    
>>> import pkg.sub_pkg1.mod1
>>> pkg.sub_pkg1.mod1.load_data()
loading data using mod1.load_data()
>>> from pkg.sub_pkg1 import mod2
>>> dir()
['__builtins__', '__doc__', '__loader__', '__name__', '__package__',
'__spec__', 'help', 'mod2', 'pkg']
>>> mod2.clean_data()
cleaning data using mod2.clean_data()
>>> from pkg.sub_pkg2.mod3 import merge_data
>>> merge_data()
merging data using mod3.merge_data()
>>> from pkg.sub_pkg2.mod4 import Winner as Result
>>> x = Result()
>>> x
<pkg.sub_pkg2.mod4.Winner object at 0x108159588>

In addition, a module in one subpackage can reference objects in a sibling subpackage (in the event that the sibling contains some functionality that you need). For example, suppose you want to import and execute load_data() (defined in module mod1) from within module mod3. You can use an absolute import:

Python
      
    
def merge_data():
    print('merging data using mod3.merge_data()')

class Message:
    pass

from pkg.sub_pkg1.mod1 import load_data
load_data()

Here’s what that looks like:

Python
      
>>> from pkg.sub_pkg2 import mod3
loading data using mod1.load_data()
>>> mod3.load_data()
loading data using mod1.load_data()

Or you can use a relative import, where .. refers to the package one level up. From within mod3.py, which is in subpackage sub_pkg2:

.. evaluates to the parent package (pkg)
..sub_pkg1 evaluates to subpackage sub_pkg1 of the parent package.

Here’s pkg/sub__pkg2/mod3.py:

Python
      
    
def merge_data():
    print('merging data using mod3.merge_data()')

class Message:
    pass

from .. import sub_pkg1
print(sub_pkg1)

from ..sub_pkg1.mod1 import load_data
load_data()

Here’s what you get:

Python
      
        
      
    
>>> from pkg.sub_pkg2 import mod3
<module 'pkg.sub_pkg1' (namespace)>
loading data using mod1.load_data()
>>> dir()
['__builtins__', '__doc__', '__loader__', '__name__', '__package__',
'__spec__', 'help', 'mod3']

00:00 In this video, you’ll explore subpackages. Packages can be nested to contain subpackages, and that can be done to an arbitrary depth. For this example, you’ll start with package pkg. Underneath that, create a subdirectory called sub_pkg1 and sub_pkg2.

00:20 And then inside sub_pkg1/, you’ll have mod1 and mod2 and in sub_pkg2—mod3 and mod4.

00:25 You’ll use the similar dot notation to access all of these. Let me have you practice it. For this example, you’re not going to use the __init__.py, so go ahead and delete that file.

00:36 And inside the pkg/ directory, create a new directory sub_pkg1, and then move mod1 and mod2 inside of it.

00:46 And then create another directory sub_pkg2, and inside sub_pkg2/ move mod3 and mod4. pkg/ has sub_pkg1/ and sub_pkg2/, which has mod1 and mod2 and mod3 and mod4 inside of them. Okay.

01:02 And as a reminder, mod1 has load_data() and Customer, mod2 has clean_data() and Location, mod3 has merge_data() and Message, and mod4 has send_mail() and Winner.

01:15 Okay. Back here in the terminal, go ahead and start your REPL. To import things, you just continue to use dot notation, except for now you have an additional level. So pkg.sub_pkg1.mod1—well, that would import

01:36 all of mod1. Customer and load_data() are both available there. It’s just extending the module in the dot notation.

01:43 You can also use the from syntax. You could import sub_pkg1.mod2—

01:53 or actually import all of it. This is it just using the different forms, right, so you can import mod1. You could say from sub_pkg1 import mod2, so mod2 is there.

02:02 It’s just going to change the level of dot notation. So in this case, mod2 is going to be all the way at the top level of the symbol table, so mod2.clean_data() is available.

02:17 Or again, you can import individual items. So pkg.sub_pkg2, you could import merge_data, so now merge_data() is here at the local symbol table.

02:30 And you can also use the

02:34 form where you’ve created an alias for it. So import Winner as Result.

02:44 And now you’ve created, using sub_pkg2.mod4, the Winner object—an object named x. So, all the techniques work, you just have to keep in mind the subpackage dot notation extending the module a little further.

02:58 A module in one subpackage can reference objects in a sibling subpackage. So something like this—you could say inside of mod3, when it’s imported, let’s say before running merge_data() by itself you needed something from sub_pkg1, you need to load_data and then call it.

03:21 So now, if you were to

03:29 import mod3—I think I’ve already imported mod3, so exit the REPL here and restart it. Otherwise it won’t reload it. We learned that earlier. Okay. Try that statement again.

03:46 Yeah, there we go. We can see it importing and running that code, and all of mod3 is available there. There’s also another form of notation that you can use.

03:56 You’ve practiced using dot notation to access the package and subpackages, but you can also use relative import, and that’s where you use the dot dot (..) and that’s going to evaluate to the parent package.

04:08 You might’ve used .. to indicate going up a level in a directory. In this case, it’s going to evaluate all the way to the parent package. And then if you wanted to access a subpackage, you use .., and then no space, sub_pkg or whatever the subpackage’s name is, and that’s going to evaluate to the subpackage of the parent package.

04:27 Let me show you what that looks like. To try out that dot notation, go back inside mod3 and change this first line to from .. import sub_pkg1.

04:39 This is going to import the entire namespace of sub_pkg1, and to show that that’s worked, go ahead and print sub_pkg1. And then from ..sub_pkg1.mod1 you could import load_data again. Again, showing the relative import here. And call it. Again, you’ve modified mod3 a little bit here, changed these two import statements to show the idea of importing from one level up, going back to the root—the sub_pkg1—and then going back up and looking at the root, which then goes inside sub_pkg1 here using dot notation and importing that single function from mod1.

05:18 That’s going to import it again at the local level.

05:23 Exiting and restarting the REPL. Now, if you were to import sub_pkg2, import mod3. And there you can see it—adding the namespace for sub_pkg1, and then also calling the function load_data().

05:38 The local symbol table—you can see a couple of things. One, you can see mod3 has load_data(), its own merge_data() and Message, and then also sub_pkg1. So sub_pkg1—you imported mod1 and its objects are there. Pretty neat. All about dot notation. All right!

05:58 You’re almost done with the course! Next up is a conclusion and course review.

danny vw on March 4, 2020

Short question: if we have an additional level, e.g., grandparent_pkg/pkg1/sub_pkg2/mod3.py and suppose I need to go via the grandparent to reach some other modules in grandparent_pkg/pkg2/sub_pkg7/mod8.py. To go up to the grandparent from within mod3.py do we use then something like ../..pkg2.sub_pkg7/mod8.py ?

Chris Bailey RP Team on March 5, 2020

Hi @danny vw, I haven’t played around much with going that far backward. But it looks like instead of adding another pair of dots with a slash, you would add just one more dot for the next level up. So it would be from ...pkg2.sub_pkg7.mod8 import your_function. Two dots take you up to the parent, three dots to the grandparent.

karenson on May 15, 2020

I don’t understand why the last example only loads mod1 from sub_pkg1? Why isn’t mod2 loaded when you use “from .. import sub_pkg1”?

Playing around, sub_pkg1.mod1 is only loaded when you mod3 contains both “from .. import sub_pkg1” and “from ..sub_pkg1.mod1 import load_data”

Can you expound on what is causing this?

Chris Bailey RP Team on May 18, 2020

Hi @karenson,

I know that looks weird. What I have learned is that 2 non-obvious things are happening here. If you were to import one of the subdirectories by themselves, import pkg.sub_pkg1, it will only import the namespace and not the whole contents of the modules within. So, you must specify the module, not just it’s parents directory.

The second and probably weirder thing that happens, is that if you import only a portion of a module, like in the examples the function load_data, the other items from that module are imported also. So that’s why you can see if you check that the class Customer is also there. If you want to try an experiment, try importing into the REPL, from pkg.sub_pkg2.mod4 import send_mail, what you should find is the class Winner came along for the ride during that import. You can see the namespace pkg.sub_pkg2.mod4 has both.

Guillermo Saldivar on Oct. 12, 2020

How is the ‘parent package’ determined by Python? Why isn’t ModulesAndPackages considered a package, if you’re running the repl from that directory/level? Is there a way to let python know I would like to consider ModulesAndPackages as that ‘parent package’ (so I can access, for example, fact.py and mod.py)

Geir Arne Hjelle RP Team on Oct. 12, 2020

@Guillermo Saldivar

Typically, your current directory is not considered a package, but rather as a directory containing modules and packages. In this example, you should be able to access fact.py by importing it directly using import fact.

Python looks for modules and packages in something called the Python import path. You can see more information about this at realpython.com/python-import/#pythons-import-path

Become a Member to join the conversation.