functools Module

Python Coding Interviews: Tips & Best Practices James Uejio 05:20

In this lesson, you’ll learn about the functools module. This module contains some useful higher order functions like reduce() and some decorators like cached_property and lru_cache.

functools.reduce() is useful to apply a function over and over on an iterable to “reduce” it to one single value:

Python
      
>>> from functools import reduce
>>> reduce(lambda x, y: x * y, [1, 2, 3, 4])
24

functools.cached_property is available in Python 3.8 and above and allows you to cache class properties. Once a property is evaluated, it won’t be evaluated again.

functools.lru_cache allows you to cache recursive function calls in a least recently used cache. This can optimize functions with multiple recursive calls like the Fibonnacci sequence.

You can check out the Python documentation on the functools module.

00:00 In this video, you’ll learn about the functools module and learn about one higher order function reduce() and two decorators, cached_property and lru_cache. Let’s get started with reduce()—from functools import reduce.

00:13 reduce() is a function that helps us combine, or reduce, a iterable into one value. For example, reduce() of lambda—a two-argument lambda—x + y, a list of

00:27 [1, 2, 3, 4]. 10. So, what happened there is it did 1 + 2,

00:36 and then did that value + 3, and then did that value + 4.

00:45 Let’s do one more example. Multiply and then pass in an initial value 10. So, that did 10 * 1—whatever the result is there * 1, * 3, and * 4.

01:04 It’s important to include those parentheses to show that it actually evaluates left to right. It doesn’t evaluate just all at once. So, in an interview, if the interviewer asks you to somehow combine a list into one value, according to some rules, then functools.reduce() should come to mind.

01:20 Let’s move on to the decorators. Let’s pretend we have a Data class that takes in an __init__(self, n). self.n = n, and then let’s define a @property which is def f(self) and does something that takes a long time. So, maybe total = 0 for i in range(self.n): for j in range(self.n): for k in range(self.n): total += i + j + k.

01:53 Then, return total. Let’s load the function interactively using IPython.

01:59 d = Data(200), d.f, d.f. It’s evaluating this n-cubed function over and over again. Notice how this function doesn’t actually have any side effects, and so we can actually cache this value by using the cached_property decorator.

02:17 from functools import cached_property. Note, this will only work in Python 3.8 and above, and that’s why my VS Code is yelling at me, because I think that VS Code linter is using Python 3.7.

02:30 Same with IPython—I set it up to use Python 3.7, and so instead I’ll use Python 3.8, and I also replaced the word @property with @cached_property. Let’s try it.

02:40 d = Data(200), d.f takes a little while—but now, it’s instant! There’s also a pip package cached-property that does the same thing, and you can use it in versions before Python 3.8.

02:52 Let’s move on to another cached decorator. Imagine you’re writing the Fibonacci function. So, the Fibonacci series looks something like this. The zeroth at the Fibonacci number is 0, the first Fibonacci is 1, the next Fibonacci number is the previous two added together, and so on.

03:09 So, the next one would be 1 + 1, 2 + 1, 3 + 2, et cetera. This is a very natural recursive function. Our base case would be if n <= 1 return n, otherwise, return fib(n - 1) + fib(n - 2).

03:31 Save it, run it interactively. Let’s just run it with Python 3.8, so the import at the top doesn’t error. fib(5)? 5. fib(10)? 55. fib(100)? Takes a long time.

03:47 Let’s see why it’s taking a long time. You can print(n),

03:53 run this, fib(5). Notice how fib(5) actually called fib(4) and fib(3).

04:06 fib(4) called fib(3) and fib(2). This call also called fib(2) and fib(1).

04:16 fib(3) called fib(2), called fib(1), and this one called fib(1) and fib(0), et cetera. It’s sort of like a tree. It’s splitting by two calls each time.

04:26 You can imagine fib(100) is doing a lot of duplicate work. This is where the lru_cache comes in. from functools import lru_cache.

04:36 This is available in Python 3.2 and above. Wrap our function in @lru_cache, save, exit. Now, when you call fib(5), it actually caches those values, and then when you have to access it later on, it can access it immediately. So, fib(100)—now, it takes a lot faster. And notice how, here, it actually stopped printing out after 6, because 5 through 1 were actually already cached, here.

05:06 That concludes the functools video. As always, there are many more functions included in the module and I’ll link the documentation down below. In the next video, you’ll learn about the doctest module and assert statements.

James Uejio RP Team on April 27, 2020

Here is the Python documentation on the itertools module: Python itertools module

James Uejio RP Team on April 27, 2020

Sorry wrong link: Here is the Python documentation on the functools module: Python functools module

reebaabu on Dec. 16, 2020

Why are we using @property here ?

James Uejio RP Team on Dec. 30, 2020

Hi @reebaabu I am using @property to show the difference between @property and @cached_property. @cached_property is the same as @property but cached so it is useful to see both!

iamrsingh on March 7, 2022

Hi, the supporting material of pdf for these lectures are of no use. They just mention the topic names and nothing else. could you guys pls update that? It will be great to have something to refer quickly for these topics.

thanks

Become a Member to join the conversation.