Registering Plugins With Decorators

I think I’ve missed something already explained. I wanted to ask about the different form of decorator here. Why does it not have include the inside wrapper function?

And now I’m wondering why we can’t do similar to the above all the time.

e.g.

Why can’t a decorator from before, such as:

def slow_down(func):
    """ Delay for 1 second """
    def wrapper_slow_down_one():
        time.sleep(1)
        return func()
    return wrapper_slow_down_one

not instead just be:

def slow_down(func):
    """ Delay for 1 second """
    time.sleep(1)
    return func

Hi Sam,

the different pattern used here for @register can be used when your decorator only needs to do something when the decorated function is defined, in this case it’s registered to a dictionary.

Since we’re not changing the function that has been passed in, that function will work exactly as if it had not been decorated. In the usual pattern, the decorated function is instead replaced by the inner wrapper function.

If you use your simplified @slow_down decorator, you will notice that the code sleeps for 1 second whenever you define a new decorated function, but it will not sleep when you call that decorated function.

Oh of course. Seems obvious now you say it. Ok, thank you.

I think there is no need to have return func in the def register(func). Correct me if I am wrong. Thanks

@levan8421 A decorator replaces the function being defined with what’s being returned from the decorator. If you don’t return func, the decorator is implicitly returning None. That would mean that the decorator would replace any function it decorates with None.

For example:

>>> def register(func):
...     print(f"registering {func}")
...
>>> @register
... def hello(name):
...     return f"Hello {name}"
...
registering <function hello at 0x7f432a989200>

>>> print(hello)
None

>>> hello("world")
TypeError: 'NoneType' object is not callable

Thank you for your response. I understand why decorator needs to return the func in general. I am just talking about this particular example of registering plugins:

import random
PLUGINS = dict()

def register(func):
    PLUGINS[func.__name__] = func

@register
def add(x):
    return x+5

def random_cal(x):
    func_name, func = random.choice(list(PLUGINS.items()))
    print(f'*** Using {func_name}')
    return func(x)

In the decorator **def register(func)**, the func is already added to PLUGINS and that is the sole purpose of this decorator. Later when we call the random_cal, the registered function will be called from the PLUGINS dictionary. That is why I think there would be no need to have *return func* in the def register(func). Of course, I would agree if it is for convention purpose.

@levan8421 True. In that particular example it works without returning the function, because you are only referencing the registered function through the PLUGINS dictionary.

However, it seems bound to cause issues later to have the decorator make the function uncallable directly, so I would still highly recommend returning func - even if your current code doesn’t use it.

Great video. LOLed on the “The func that comes in” :D

This example made me understand something I did not grasp at first sight: it looks like the decorator gets executed when we define the decorated function.

(I thought the decorator was executed when we executed the decorated function.)

@Bastian, right. The decorator (meaning the function with the exact same name used behind the @ when decorating) is executed when the decorated function is defined.

In the most typical decorator pattern with the inner function, this decorator then defines a new (inner) wrapper function that is executed when the decorated function is executed. But simple decorators like the one in this video only run on “define-time”.

Ho yes thanks @Geir Arne Hjelle for reminding me that a typical decorator pattern is executed only to define a new wrapper function. This is all so logic and yet feels so counter intuitive to me. I think I get it technically but I still need to get around the concepts and use cases.

In the last real-world example, the decorated functions become elements of a dictionary and this is described as “registering plugins”. I cannot understand where this applies, thus becoming a real world example.

@Konstantinos Plug-ins gives you an architecture where you can more easily “dynamically” decided which function to call. I use this a lot in my own real world code both for flexibility and because it keeps my code simple.

For a concrete example, pyconfs is my library for working with configurations. It separates “configuration” from “file format” for flexibility. In other words, most of the library doesn’t care or even know whether it’s working with TOML, YAML, or JSON.

This is done by defining readers as plug-ins that are used to load the configuration into memory. If you want to support a new input format, that’s as easy as writing the function that parses that format into memory and decorating that function with @register. You don’t need to change any other part of the code, because the rest of the code only works with the dictionary of registered functions.

The same pattern can be used in many other real world examples. For example, you can register different models to use when calculating satellite orbits, different stages to use in a machine learning model, different windows or tabs to expose in a GUI, and so on.

Another example could pull platform specific code into plug-ins, so that you’d call some code on Windows and another on Linux. Again, this would allow you to add support for new platforms without needing to change the existing code.

I talk a bit more about this principle in this presentation: pyvideo.org/pycon-us-2019/plugins-adding-flexibility-to-your-apps.html

Thank you @Geir Arne Hjelle for answering that in detail. So, in a high level, I understand that the registered functions should always have the same objective such as “read-file-formats”, “calculate-satellite-orbits”, etc that vary in implementation. It seems like an interesting technique, but needs some practice in my real world situations.

@Konstantinos Yes, exactly. It typically works most efficiently when grouping together functions with the same purpose. Usually you want your registered functions to even share the same signature so that you don’t need “extra knowledge” to call them.

@Geir Arne Hjelle Thank you very much for your concise and clear explanation.