Covering Lazy Imports

00:00 In the previous lesson, I gave an overview of the course. In this lesson, I will cover the new lazy import feature. Up until now, import statements at the top of a module would get run when that module was loaded. Modules often load other modules, so that meant the whole import tree would typically get run unless a clever programmer did something special.

00:20 There are certain types of code where this can get expensive. Consider a command-line tool with lots of different options, each one of which needs its own submodule.

00:30 When the user runs the tool, they only need one of these options, yet all of the submodules need to get loaded. This can be expensive. To deal with this problem, Python 3.15 introduces lazy loading.

00:44 By marking an import with the soft keyword lazy, you delay the loading of the module until its contents are used. If you’re not familiar, a soft keyword is one that can be used for other things, so if you had a variable named lazy in your existing code, it will still work as a variable. Lazy loading can drastically improve the startup time of some programs, especially that subcommand-style tool example I just gave.

01:09 Let’s go look at how lazy loading works in the REPL. Consider this toy program named normal.py. This is the non-lazy variation of a script that loads some modules and then calls a function.

01:23 To illustrate how the new keyword works, I’ve littered the code with print statements. Say I was writing one of those command-line tools I spoke about that converted some content into either RTF or HTML format.

01:36 I might have a module containing an RTF conversion function and another one for HTML. In the body of my program, I’m printing out when importing is complete, then telling you when I’m about to call a converter. In a real version of this kind of program, there would be some logic here typically using argparse to figure out what subcommand to call.

01:58 Instead, I’m just hard-coding a call to convert_rtf().

02:03 This is the rtf module. At the top, it tells you when the load of the module is invoked. Then I define a conversion function that prints out a message pretending to convert a value.

02:15 This is the equivalent html module. Same kind of print up top, same kind of conversion function inside.

02:24 I’ve put normal.py back in the top screen here so you can see it while I run it. Let’s give it a go.

02:34 I’ve colorized the output to highlight where each print statement comes from. The blue lines are from normal.py. When it loads, it first prints out a message on line 2 above, then it imports the rtf module.

02:48 Loading that module has the side effect of printing out a message, which here I’m showing in green. Next, normal.py loads the html module, which also prints out a message, here shown in the orange-y burnt something or other color. Once the modules are loaded, the prints on line 6 and 7 get run, which are shown here in blue.

03:08 Then finally, line 8 calls the RTF converter. The results in green text are the prints from the conversion function at the bottom. Let’s revisit this using the new lazy keyword.

03:22 This is sloth.py. Well, what else should you name some lazy code? Like normal.py, it starts out with a print statement. Then here is the new lazy soft keyword.

03:35 The rest of the line looks like a regular import. It just now has lazy as a prefix. And then the same idea for our html module.

03:44 I’ve updated the print messages to say sloth, but otherwise this is the same as before. Let’s give it a run.

03:53 I used the same colors as before. The first thing you might notice is that the orange burnt html is gone. sloth.py never actually uses the html module, and since it was lazy loaded, it never actually gets loaded.

04:07 So its print side effect doesn’t happen. There’s something a little more subtle going on here as well. The order of the lines is different. In the previous run, the imports done and calling converter output from lines 6 and 7 got called after the print side effect of loading rtf.

04:26 Because of the lazy loading, that side effect doesn’t happen until the actual use of convert_rtf(). So you get nothing but blue until line 8, which then triggers the side effect of the load as well as the output of the function.

04:42 If this code had a lot of other modules, or if loading html was particularly expensive, all that cost would be deferred, or in our case, not triggered at all.

04:52 Another possible cost savings is when dealing with type checking. Sometimes type checking requires you to load things you don’t need except when checking, and that is typically not required when running the program. To deal with this, you often see an if TYPE_CHECKING block loading the required asset only if the code was being type checked.

05:12 Now you can skip that and just lazy load the object. If you’re in type checker mode, the type checker will have the definition as necessary without your code needing to load it.

05:23 Similarly, lazy loading can also resolve certain circular import cases where module A needs B and B needs A. This doesn’t solve the problem when that need is a side effect of both modules, but it does solve the problem where something from B gets used inside of a function of A.

05:41 By lazy loading B, the module won’t get loaded until the actual use, solving the circular import. That’s nice because it means you can stick your imports back at the top of the file where they belong, rather than in nested code where they had to hide before to solve this issue.

05:57 In addition to the soft keyword, you can also use __lazy_modules__. This works like __all__ in that you give it a collection of names of modules to lazy load. Putting this variable above your imports causes the imports mentioned in the variable to be lazy loaded.

06:13 Why would you do it this way? Well, since it’s just a variable, older versions of Python will treat it like a variable and just go about the normal loading process.

06:23 Python 3.15 recognizes it and uses the lazy mechanism for the named modules. This means you can avoid code that checks for the Python version and still take advantage of lazy loading if you’re running in 3.15.

06:36 Additionally, you can also see which modules are targeted for lazy loading by inspecting the new sys.lazy_modules attribute. There are a couple of downsides.

06:47 Errors in lazy loaded modules won’t show up until they’re loaded. This means failures may wait until invocation time rather than at program start. Also, sometimes the side effects of a module are important. For example, in a registration pattern, a module adds itself to a plugin registry.

07:05 You have to be careful not to lazy load that kind of module.

07:10 Next up, two new features, unpacking in comprehensions and frozen dictionaries.