Caching and Reloading

00:00 In the previous lesson, I showed you namespace packages. In this lesson, I’m going to introduce you to the module cache. Importing a module can be a relatively expensive operation, and as such, Python wants to make sure that it only does it once.

00:14 Importing is typically done at the top of a script, which means getting all the imports to happen can be perceived as a startup cost. The user doesn’t see their program get going until all the importing is done.

00:27 As such, Python caches modules once they’re imported. This means subsequent calls to import for the same module return the cached copy. This makes importing much faster.

00:38 It isn’t just about your libraries, but about your library’s libraries. For example, during the version 2 to version 3 transition, it was common to find a library called six, which helped scripts operate in both Python 2 and Python 3 modes.

00:52 As such, almost every package out there used it. If you used 4 or third-party libraries, in all likelihood, you’d be importing the six library four times, five if your own script had to handle both versions as well.

01:05 There’s a subtle point I need to make. Remember when I recommended that you only import the thing from a module that you’re actually going to use and how it keeps your namespace clean?

01:14 Well, the key part of that sentence is namespace. Whether you import one thing or everything from a module, the module still gets loaded and cached. There’s a distinction between what’s in the module cache versus what’s in the namespace.

01:29 Let’s go look at what this means in practice.

01:33 It’s an oldie but a goodie. Let me start by importing pi from the math module,

01:40 and there it is, which means pi is in the namespace, but seeing as I only imported pi from math, it is the only thing that got added to the namespace.

01:50 If I want to get the constant e, I have to import that as well. Meanwhile, the cos function from the math module isn’t there.

02:01 If I try to use it, I get an error. Since I used the from math format math isn’t in the namespace either, so I can’t get at cos that way. With me so far?

02:15 Alright, let’s look under the hood. Like with the module path, the module cache is available in sys. Logically enough, it’s called modules and it’s a dictionary with keys being the name of the modules.

02:32 Here you see that math is in the cache, even though I only loaded pi and e into the namespace, and I can prove that by

02:44 actually calling cos directly from the cached module. This isn’t recommended. If you want cos, you should just import it, but you can see that Python has math in the cache because I imported something from it.

02:59 One of the consequences of having a cache is well, things are in the cache. If you make a change to code in one of your files, that change won’t show up even if you call import again. The cached module is what gets used.

03:12 There’s a way around this though. The built-in importlib module has tools for importing, and one of them is reload().

03:22 I seem to have had a habit in this course of using very small files. It says dogs.py, and it contains a list of famous dogs. Now to the REPL.

03:33 Imported, and there’s the list. Nothing special so far. I’m going to go back to dogs.py and make an edit. Did you blink? I added Lassie on the end.

03:47 Back in the same REPL.

03:51 Nope, no Lassie. What if I import it again?

03:59 Nope. Still no Lassie. This is that module cache. The old version is in the cache, so that’s all you get. You can get around this using the reload() tool in the importlib module.

04:17 Note that what I’m passing to the reload() function is the module itself, not a string with the name of the module. If I had used from dogs import famous, I’d have to go and import dogs to get a reference to the module, or I could use the copy in the sys.modules cache.

04:35 Now that I’ve reloaded it, there’s Lassie. Just in time too. Timmy went and fell down a well again. Having the module cache means two packages can reference each other in a situation called circular imports.

04:50 In the next lesson, I’ll show you when that does and when that doesn’t work.