Shadowing Modules

00:00 In the previous lesson, I showed you the various ways submodules and subpackages can work when you import their parents. In this lesson, I’ll show you what happens if you load two modules with the same name.

00:11 Variables in Python aren’t like variables in some languages. They aren’t a box containing something, they’re a reference to an object. When you construct a string or create an integer, the variable you’re defining is actually a reference to a string object or an integer object.

00:27 At any time, you can create a new reference in the namespace that has the same name as the old reference, but points to a different object. This overrides the original definition, sometimes known as “shadowing” it. For simple variables, the thing being shadowed can’t be recovered unless you squirreled it away in another reference.

00:46 But you can shadow built-in objects like functions. You can redefine print() if you really want to. If you do shadow print(), you can then get the original back by deleting your reference.

00:58 Python lets you override built-ins, but doesn’t let you get rid of them, so you can’t delete the original print(). Everything in Python is an object, and if I had a nickel for each time I’ve said that I’d have at least one nickel, wouldn’t I?

01:11 Modules are no different. They’re objects too, which means they also can be shadowed … most of the time. Let’s go play in the shadows.

01:21 I’ve created a Python file named random.py. Inside, it has a single function that returns a random number. It isn’t a very good random number. That’s not the point.

01:31 In case you’re not aware, Python standard library has a module with the exact same name as this file. Let’s import this. Python doesn’t care. It happily imports my local copy.

01:44 I’m thinking of an integer between 1.9 and 2.1. Can you guess it? How’d it do that?

01:53 See? No problems calling get_random() and getting its response.

01:57 The standard library’s random module has a function called randint(). Let me import that,

02:06 and that’s shadowing in a nutshell. Python will only ever load a module once, so by loading my local version of random, it’s defining that module in the namespace.

02:17 When I try to import something from the standard library’s random module, it sees random as the existing module, the local one in the namespace. As it only has get_random() and no randint(), this is problematic.

02:31 Thankfully, the error message is much improved. In older versions of Python, it simply told you there was no such thing to import, and so if there was a misnamed Python file in your directory shadowing a built-in, you could pull your hair out trying to figure out why a standard library function couldn’t be found.

02:46 I’ve had this happen to me enough times that I finally learned the likely culprit. The new version of the error includes the name of the module file, making it much more obvious as to what’s going on.

02:56 Score one for improved error messages.

03:00 Not all modules can be shadowed. Let’s look at another one. In the top window, I’ve created time.py. Like with random.py, this has the same name as a built-in module.

03:12 Let’s load this one. There’s the import.

03:18 Ooh, wait a second. Where’d my function go? Time can’t be shadowed. Unfortunately, this is that error I was just complaining about, but in the other direction.

03:27 It can leave you scratching your head. Why can’t it find my function? It’s right there. Poor developer could go bald. Let’s look at the difference between these two modules to see what’s going on.

03:40 Evaluating my local random module in the REPL shows you that it’s a source file, whereas time tells you it’s built in. Through the magic of video editing, I’ve exited the REPL, started a new one and tacked it onto the bottom here so you can see it.

03:57 I also changed directories so that my shadowy random.py isn’t there anymore. To see the difference between time and the standard library’s real random, let’s import it and evaluate it.

04:11 That’s a very long chunk of output thanks to how Python gets installed on macOS. The important thing to notice is that it ends in random.py. Modules in Python’s standard library can be Python files, just like your Python files, but they can also be pre-compiled binaries.

04:29 Time is a pre-compiled binary, so it shows up as built-in. Pre-compiled binaries get looked for first when you call import, whereas those Python file types, they get looked for in the order specified by the module search path.

04:45 The module search path includes the current directory before the system directory, so it finds my local random.py and allows the shadowing, so that’s why random can be shadowed, but time cannot.

04:57 I’ll show you more on the system path that tells Python where to look for modules in a later lesson.

05:03 To share packages with others, they can be bundled up and made installable. In the next lesson, I’ll show you the structure of just that kind of package.