Whenever you perform calculations in Python, you make use of built-in operators such as +, %, and **. Did you know that Python also provides an operator module? While it may seem that the purpose of operator is to provide an alternative to these existing Python operators, the module actually has a far more specialized purpose than this.
The Python operator module provides you with a set of functions, many of which correspond to the built-in operators but don’t replace them. The module also provides additional functionality, as you’ll soon discover.
In this tutorial, you’ll learn how to:
- Use any of the basic operator-equivalent functions
- Pass
operatorfunctions as arguments - Serialize
operatorfunctions for later use - Gauge
operatorfunction performance against common alternatives - Use higher-order
operatorfunctions in a range of interesting example cases
To get the most out of this tutorial, you should be familiar with the basic Python operators and the functional programming idea of one function returning another function back to its caller.
With that knowledge, you’re ready to dive in and learn how to use the greatly misunderstood operator module to your advantage.
Get Your Code: Click here to download the free sample code that shows you how to use Python’s operator module.
Using the Python operator Module’s Basic Functions
In this section, you’ll learn about the operator module’s operator-equivalent functions that mimic built-in operators, and you’ll pass them as arguments to higher-order functions. You’ll also learn how to save them for later use. Finally, you’ll investigate the performance of the operator-equivalent functions and uncover why you should never use them where a built-in Python operator will do instead.
Learning How the Basic Functions Work
The Python operator module contains over forty functions, many of which are equivalent to the Python operators that you’re already familiar with. Here’s an example:
>>> import operator
>>> operator.add(5, 3) # 5 + 3
8
>>> operator.__add__(5, 3) # 5 + 3
8
Here, you add 5 and 3 together using both add() and __add__(). Both produce the same result. On the face of it, these functions provide you with the same functionality as the Python + operator, but this isn’t their purpose.
Note: Most of the operator module functions contain two names, a dunder version and a without-dunder version. In the previous example, operator.__add__(5, 3) is the dunder version because it includes double underscores. From this point forward, you’ll use only the without-dunder versions, such as operator.add(5, 3). The dunder versions are for backward compatibility with the Python 2 version of operator.
If you take a look at the list of functions that operator provides for you, then you’ll discover that they cover not only the arithmetic operators, but also the equality, identity, Boolean, and even bitwise operators. Try out a random selection of them:
>>> operator.truediv(5, 2) # 5 / 2
2.5
>>> operator.ge(5, 2) # 5 >= 2
True
>>> operator.is_("X", "Y") # "X" is "Y"
False
>>> operator.not_(5 < 3) # not 5 < 3
True
>>> bin(operator.and_(0b101, 0b110)) # 0b101 & 0b110
'0b100'
In the code above, you work with a selection from the five main categories. First, you use the equivalent of an arithmetic operator, and then you try out equality and identity operator examples in the second and third examples, respectively. In the fourth example, you try a Boolean logical operator, while the final example uses a bitwise operator The comments show the equivalent Python operators.
Before reading the rest of this tutorial, feel free to take some time to experiment with the range of operator-equivalent functions that Python’s operator module provides for you. You’ll learn how to use them next.
Passing Operators as Arguments Into Higher-Order Functions
You use the operator-equivalent functions most commonly as arguments for higher-order functions. You could write a higher-order function that performs a series of different tasks depending on the operator function passed to it. Suppose, for example, you wanted a single function that could perform addition, subtraction, multiplication, and division. One messy way of doing this would be to use an if … elif statement as follows:
>>> def perform_operation(operator_string, operand1, operand2):
... if operator_string == "+":
... return operand1 + operand2
... elif operator_string == "-":
... return operand1 - operand2
... elif operator_string == "*":
... return operand1 * operand2
... elif operator_string == "/":
... return operand1 / operand2
... else:
... return "Invalid operator."
...
In your perform_operation() function, the first parameter is a string representing one of the four basic arithmetic operations. To test the function, you pass in each of the four operators. The results are what you’d expect:
>>> number1 = 10
>>> number2 = 5
>>> calculations = ["+", "-", "*", "/"]
>>> for op_string in calculations:
... perform_operation(op_string, number1, number2)
...
15
5
50
2.0
This code is not only messy, but also limited to the four operators defined in the elif clauses. Try, for example, passing in a modulo operator (%), and the function will return an "Invalid operator" message instead of the modulo division result that you were hoping for.
This is where you can make excellent use of the operator functions. Passing these into a function gives you several advantages:
>>> def perform_operation(operator_function, operand1, operand2):
... return operator_function(operand1, operand2)
...
This time, you’ve improved your perform_operation() function so that the first parameter can accept any of the operator module’s functions that take exactly two arguments. The second and third parameters are those arguments.
The revised test code is similar to what you did before, except you pass in operator functions for your perform_operation() function to use:
>>> from operator import add, sub, mul, truediv
>>> number1 = 10
>>> number2 = 5
>>> calculations = [add, sub, mul, truediv]
>>> for op_function in calculations:
... perform_operation(op_function, number1, number2)
...
15
5
50
2.0
This time, your calculations list contains references to the functions themselves. Note that you pass in function names and not function calls. In other words, you pass in add to perform_operation(), and not add(). You’re passing in the function object, not the result of its execution. Remember, the name of a function is actually a reference to its code. Using the () syntax calls the function.
There are two advantages to using your updated version of perform_operation(). The first is expandability. You can use the revised code with any of the other operator functions that require exactly two arguments. Indeed, you might like to experiment by passing the operator module’s mod(), pow(), and repeat() functions to both versions of your function. Your updated version works as expected, while your original version returns "Invalid operator".
The second advantage is readability. Take a look at both versions of your perform_operation() function, and you’ll notice that your second version is not only significantly shorter, but also more readable, than the original.
Passing functions as arguments to other functions is a feature that you’ll often use in functional programming. This is one of the main purposes of the operator module. You’ll study other examples of this later.
Serializing operator Module Functions
One way of saving objects, including functions, to disk is to serialize them. In other words, your code converts them into byte streams and stores them on disk for later use. Conversely, when you read serialized objects back from disk, you deserialize them, allowing them to be read from disk into a program for use.
There are several reasons why you might serialize functions, including to save them for future use in another program or to pass them between different processes running on one or more computers.
A common way to serialize functions in Python is by using the pickle module. This, along with its dictionary wrapper shelve, provides one of the most efficient ways of storing data. However, when you serialize a function using pickle, then you only serialize its fully qualified name, not the code in the function body. When you deserialize a function, the environment must provide access to the function’s code. The function can’t work otherwise.
To see an example, you’ll revisit the earlier perform_operation() example. You’ll call different operator functions to perform the different operations. The following code adds a dictionary that you’ll use to map a string operator to its matching function:
>>> import operator
>>> operators = {
... "+": operator.add,
... "-": operator.sub,
... "*": operator.mul,
... "/": operator.truediv,
... }
>>> def perform_operation(op_string, number1, number2):
... return operators[op_string](number1, number2)
...
>>> perform_operation("-", 10, 5)
5
The operations supported by perform_operation() are the ones defined in operators. As an example, you run the "-" operation, which calls operator.sub() in the background.
One way to share the supported operators between processes is to serialize the operators dictionary to disk. You can do this using pickle as follows:
>>> import pickle
>>> with open("operators.pkl", mode="wb") as f:
... pickle.dump(operators, f)
...
You open a binary file for writing. To serialize operators, you call pickle.dump() and pass the structure that you’re serializing and the handle of the destination file.
This creates the file operators.pkl in your local working directory. To demonstrate how to reuse operators in a different process, restart your Python shell and load the pickled file:
>>> import pickle
>>> with open("operators.pkl", mode="rb") as f:
... operators = pickle.load(f)
...
>>> operators
{'+': <built-in function add>, '-': <built-in function sub>,
'*': <built-in function mul>, '/': <built-in function truediv>}
Firstly, you import pickle again and reopen the binary file for reading. To read the operator structure, you use pickle.load() and pass in the file handle. Your code then reads in the saved definition and assigns it to a variable named operators. This name doesn’t need to match your original name. This variable points to the dictionary that references the different functions, assuming they’re available.
Note that you don’t need to explicitly import operator, although the module needs to be available for Python to import in the background.
You can define perform_operation() again to see that it can refer to and use the restored operators:
>>> def perform_operation(op_string, number1, number2):
... return operators[op_string](number1, number2)
...
>>> perform_operation("*", 10, 5)
50
Great! Your code handles multiplication as you’d expect.
Now, there’s nothing special about operator supporting pickling of functions. You can pickle and unpickle any top-level function, as long as Python is able to import it in the environment where you’re loading the pickled file.
However, you can’t serialize anonymous lambda functions like this. If you implemented the example without using the operator module, then you’d probably define the dictionary as follows:
>>> operators = {
... "+": lambda a, b: a + b,
... "-": lambda a, b: a - b,
... "*": lambda a, b: a * b,
... "/": lambda a, b: a / b,
... }
The lambda construct is a quick way to define simple functions, and they can be quite useful. However, because pickle doesn’t serialize the function body, only the name of the function, you can’t serialize the nameless lambda functions:
>>> import pickle
>>> with open("operators.pkl", mode="wb") as f:
... pickle.dump(operators, f)
...
Traceback (most recent call last):
...
PicklingError: Can't pickle <function <lambda> at 0x7f5b946cfba0>: ...
If you try to serialize lambda functions with pickle, then you’ll get an error. This is a case where you can often use operator functions instead of lambda functions.
Look back at your serialization code and notice that you imported both operator and pickle, while your deserialization code imported only pickle. You didn’t need to import operator because pickle did this automatically for you when you called its load() function. This works because the built-in operator module is readily available.
Investigating operator Function Performance Against the Alternatives
Now that you have an idea of how to use the operator-equivalent functions, you may wonder if you should use them instead of either the Python operators or lambda functions. The answer is no to the first case and yes to the second. The built-in Python operators are always significantly faster than their operator module counterparts. However, the operator module’s functions are faster than lambda functions, and they’re more readable as well.
If you wish to time the operator module’s functions against their built-in or lambda equivalents, then you can use the timeit module. The best way to do this is to run it directly from the command line:
PS> python -m timeit "(lambda a, b: a + b)(10, 10)"
5000000 loops, best of 5: 82.3 nsec per loop
PS> python -m timeit -s "from operator import add" "add(10, 10)"
10000000 loops, best of 5: 24.5 nsec per loop
PS> python -m timeit "10 + 10"
50000000 loops, best of 5: 5.19 nsec per loop
PS> python -m timeit "(lambda a, b: a ** b)(10, 10)"
1000000 loops, best of 5: 226 nsec per loop
PS> python -m timeit -s "from operator import pow" "pow(10, 10)"
2000000 loops, best of 5: 170 nsec per loop
PS> python -m timeit "10 ** 10"
50000000 loops, best of 5: 5.18 nsec per loop
The above PowerShell session uses the timeit module to compare the performance of various implementations of addition and exponentiation. Your results show that for both operations, the built-in operator is fastest, with the operator module function only outperforming the lambda function. The actual time values themselves are machine-specific, but their relative differences are significant.
Note: Python’s timeit module allows you to time small pieces of your code. You usually invoke timeit from the command line using python -m timeit followed by a string containing the command that you want to measure. You use the -s switch to indicate code that you want to run once just before the timing begins. In the example above, you used -s to import pow() and add() from the operator module before timing your code.
Go ahead and try the other operator functions out for yourself. Although the exact timings will vary from machine to machine, their relative differences will still show that built-in operators are always faster than the operator module equivalents, which are always faster than lambda functions.
Now you’re familiar with the operator-equivalent functions from the operator module, but you might want to spend some time exploring the rest of these functions. Once you’re ready to move on, keep reading to investigate some of the other ways to use operator.
Using the Python operator Module’s Higher-Order Functions
In this section, you’ll learn about three of the higher-order functions that Python’s operator module makes available to you: itemgetter(), attrgetter(), and methodcaller(). You’ll learn how these allow you to work with Python collections in a range of useful ways that encourage a functional style of Python programming.
Selecting Values From Multidimensional Collections With itemgetter()
The first function that you’ll learn about is operator.itemgetter(). In its basic form, you pass it a single parameter that represents an index. Then itemgetter() returns a function that, when passed a collection, returns the element at that index.
To begin with, you create a list of dictionaries:
>>> musician_dicts = [
... {"id": 1, "fname": "Brian", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 2, "fname": "Carl", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 3, "fname": "Dennis", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 4, "fname": "Bruce", "lname": "Johnston", "group": "Beach Boys"},
... {"id": 5, "fname": "Hank", "lname": "Marvin", "group": "Shadows"},
... {"id": 6, "fname": "Bruce", "lname": "Welch", "group": "Shadows"},
... {"id": 7, "fname": "Brian", "lname": "Bennett", "group": "Shadows"},
... ]
Each dictionary contains a record of a musician belonging to one of two groups, the Beach Boys or the Shadows. To learn how itemgetter() works, suppose you want to select a single item from musician_dicts:
>>> import operator
>>> get_element_four = operator.itemgetter(4)
>>> get_element_four(musician_dicts)
{"id": 5, "fname": "Hank", "lname": "Marvin", "group": "Shadows"}
When you pass itemgetter() an index of 4, it returns a function, referenced by get_element_four, that returns the element at index position 4 in a collection. In other words, get_element_four(musician_dicts) returns musician_dicts[4]. Remember that list elements are indexed starting at 0 and not 1. This means itemgetter(4) actually returns the fifth element in the list.
Next suppose you want to select elements from positions 1, 3, and 5. To do this, you pass itemgetter() multiple index values:
>>> get_elements_one_three_five = operator.itemgetter(1, 3, 5)
>>> get_elements_one_three_five(musician_dicts)
({"id": 2, "fname": "Carl", "lname": "Wilson", "group": "Beach Boys"},
{"id": 4, "fname": "Bruce", "lname": "Johnston", "group": "Beach Boys"},
{"id": 6, "fname": "Bruce", "lname": "Welch", "group": "Shadows"})
Here itemgetter() creates a function that you use to find all three elements. Your function returns a tuple containing the results.
Now suppose you want to list only the first and last name values from the dictionaries at index positions 1, 3, and 5. To do this, you pass itemgetter() the "fname" and "lname" keys:
>>> get_names = operator.itemgetter("fname", "lname")
>>> for musician in get_elements_one_three_five(musician_dicts):
... print(get_names(musician))
...
("Carl", "Wilson")
("Bruce", "Johnston")
("Bruce", "Welch")
This time, itemgetter() provides a function for you to get the values associated with the "fname" and "lname" keys. Your code iterates over the tuple of dictionaries returned by get_elements_one_three_five() and passes each one to your get_names() function. Each call to get_names() returns a tuple containing the values associated with the "fname" and "lname" dictionary keys of the dictionaries at positions 1, 3, and 5 of musician_dicts.
Two Python functions that you may already be aware of are min() and max(). You can use these to find the lowest and highest elements in a list:
>>> prices = [100, 45, 345, 639]
>>> min(prices)
45
>>> max(prices)
639
In the above code, you’ve gotten the lowest and highest values: min() returns the cheapest item, while max() returns the most expensive.
The min() and max() functions contain a key parameter that accepts a function. If you create the function using itemgetter(), then you can use it to instruct min() and max() to analyze specific elements within a list of lists or dictionaries. To explore this, you first create a list of lists of musicians:
>>> musician_lists = [
... [1, "Brian", "Wilson", "Beach Boys"],
... [2, "Carl", "Wilson", "Beach Boys"],
... [3, "Dennis", "Wilson", "Beach Boys"],
... [4, "Bruce", "Johnston", "Beach Boys"],
... [5, "Hank", "Marvin", "Shadows"],
... [6, "Bruce", "Welch", "Shadows"],
... [7, "Brian", "Bennett", "Shadows"],
... ]
The content of musician_lists is identical to musician_dicts, except each record is in a list. This time, suppose you want to find the list elements with the lowest and highest id values:
>>> get_id = operator.itemgetter(0)
>>> min(musician_lists, key=get_id)
[1, "Brian", "Wilson", "Beach Boys"]
>>> max(musician_lists, key=get_id)
[7, "Brian", "Bennett", "Shadows"]
You first create a function using itemgetter() to select the first element from a list. You then pass this as the key parameter of min() and max(). The min() and max() functions will return to you the lists with the lowest and highest values in their index 0 positions, respectively.
You can do the same with a list of dictionaries by using itemgetter() to create a function that selects key names. Suppose you want the dictionary that contains the musician whose last name comes first in the alphabet:
>>> get_lname = operator.itemgetter("lname")
>>> min(musician_dicts, key=get_lname)
{"id": 7, "fname": "Brian", "lname": "Bennett", "group": "Shadows"}
This time, you set up an itemgetter() function that selects the "lname" dictionary key. You then pass this as the min() function’s key parameter. The min() function returns the dictionary with the lowest value of "lname". The "Bennett" record is the result. Why not retry this with max()? Try predicting what will happen before running your code to check.
Sorting Multidimensional Collections With itemgetter()
In addition to selecting specific elements, you can use the function from itemgetter() as the key parameter to sort data. One of the common Python functions that you may have already used is sorted(). The sorted() function creates a new sorted list:
>>> star_wars_movies_release_order = [4, 5, 6, 1, 2, 3, 7, 8, 9]
>>> sorted(star_wars_movies_release_order)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
Now the elements of the new list are in ascending order. If you wanted to sort the list in place, then you could use the .sort() method instead. As an exercise, you might like to try it.
It’s also possible to use itemgetter() to sort lists. This allows you to sort multidimensional lists by specific elements. To do this, you pass an itemgetter() function into the sorted() function’s key parameter.
Consider once more your musician_lists:
>>> musician_lists = [
... [1, "Brian", "Wilson", "Beach Boys"],
... [2, "Carl", "Wilson", "Beach Boys"],
... [3, "Dennis", "Wilson", "Beach Boys"],
... [4, "Bruce", "Johnston", "Beach Boys"],
... [5, "Hank", "Marvin", "Shadows"],
... [6, "Bruce", "Welch", "Shadows"],
... [7, "Brian", "Bennett", "Shadows"],
... ]
You’ll now sort this list using itemgetter(). To begin with, you decide to sort the list elements in descending order by id value:
>>> import operator
>>> get_id = operator.itemgetter(0)
>>> sorted(musician_lists, key=get_id, reverse=True)
[[7, "Brian", "Bennett", "Shadows"],
[6, "Bruce", "Welch", "Shadows"],
[5, "Hank", "Marvin", "Shadows"],
[4, "Bruce", "Johnston", "Beach Boys"],
[3, "Dennis", "Wilson", "Beach Boys"],
[2, "Carl", "Wilson", "Beach Boys"],
[1, "Brian", "Wilson", "Beach Boys"]
]
To do this, you use itemgetter() to create a function that will select index position 0, the musician’s id. You then call sorted() and pass it musician_lists plus the reference to the function from itemgetter() as its key. To ensure the sort is in descending order, you set reverse=True. Now your list is sorted in descending order by id.
It’s also possible to perform more complex sorting of multidimensional lists. Suppose you want to sort each list in reverse order within musician_lists. First, you sort by last name, then you sort any lists with the same last name by first name. In other words, you’re sorting on first name within last name:
>>> get_elements_two_one = operator.itemgetter(2, 1)
>>> sorted(musician_lists, key=get_elements_two_one, reverse=True)
[[3, "Dennis", "Wilson", "Beach Boys"],
[2, "Carl", "Wilson", "Beach Boys"],
[1, "Brian", "Wilson", "Beach Boys"],
[6, "Bruce", "Welch", "Shadows"],
[5, "Hank", "Marvin", "Shadows"],
[4, "Bruce", "Johnston", "Beach Boys"],
[7, "Brian", "Bennett", "Shadows"]]
This time, you pass itemgetter() two arguments, positions 2 and 1. Your list is sorted in reverse alphabetical order, first by last name (2), then by first name (1) where applicable. In other words, the three Wilson records appear first, with Dennis, Carl, and Brian in descending order. Feel free to rerun this code and use it to select other fields. Try to predict what will happen before running your code to test your understanding.
The same principles also apply to dictionaries, provided you specify the keys whose values you wish to sort. Again, you can use the musician_dicts list of dictionaries:
>>> musician_dicts = [
... {"id": 1, "fname": "Brian", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 2, "fname": "Carl", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 3, "fname": "Dennis", "lname": "Wilson", "group": "Beach Boys"},
... {"id": 4, "fname": "Bruce", "lname": "Johnston", "group": "Beach Boys"},
... {"id": 5, "fname": "Hank", "lname": "Marvin", "group": "Shadows"},
... {"id": 6, "fname": "Bruce", "lname": "Welch", "group": "Shadows"},
... {"id": 7, "fname": "Brian", "lname": "Bennett", "group": "Shadows"},
... ]
>>> get_names = operator.itemgetter("lname", "fname")
>>> sorted(musician_dicts, key=get_names, reverse=True)
[{"id": 3, "fname": "Dennis", "lname": "Wilson", "group": "Beach Boys"},
{"id": 2, "fname": "Carl", "lname": "Wilson", "group": "Beach Boys"},
{"id": 1, "fname": "Brian", "lname": "Wilson", "group": "Beach Boys"},
{"id": 6, "fname": "Bruce", "lname": "Welch", "group": "Shadows"},
{"id": 5, "fname": "Hank", "lname": "Marvin", "group": "Shadows"},
{"id": 4, "fname": "Bruce", "lname": "Johnston", "group": "Beach Boys"},
{"id": 7, "fname": "Brian", "lname": "Bennett", "group": "Shadows"}
]
This time, you pass itemgetter() the "fname" and "lname" keys. The output is similar to what you got previously, except it now contains dictionaries. While you created a function that selected the index elements 2 and 1 in the previous example, this time your function selects the dictionary keys "lname" and "fname".
Retrieving Attributes From Objects With attrgetter()
Next you’ll learn about the operator module’s attrgetter() function. The attrgetter() function allows you to get an object’s attributes. The function accepts one or more attributes to be retrieved from an object, and it returns a function that will return those attributes from whatever object you pass to it. The objects passed to attrgetter() don’t need to be of the same type. They only need to contain the attribute that you want to retrieve.
To understand how attrgetter() works, you’ll first need to create a new class:
>>> from dataclasses import dataclass
>>> @dataclass
... class Musician:
... id: int
... fname: str
... lname: str
... group: str
...
You’ve created a data class named Musician. Your data class’s primary purpose is to hold data about different musician objects, although it could also contain methods, as you’ll discover later. The @dataclass decorator allows you to define the attributes directly by specifying their names and a type hint for their data types. Your class contains four attributes that describe a musician.
Note: You may be wondering where .__init__() has gone. One of the benefits of using a data class is that there’s no longer a need for an explicit initializer. To create an object, you pass in values for each of the attributes in the class. In the Musician class above, the first attribute gets assigned to .id, the second to .fname, and so on.
Next, you need a list of objects to work with. You’ll reuse musician_lists from earlier and use it to generate a list of objects named group_members:
>>> musician_lists = [
... [1, "Brian", "Wilson", "Beach Boys"],
... [2, "Carl", "Wilson", "Beach Boys"],
... [3, "Dennis", "Wilson", "Beach Boys"],
... [4, "Bruce", "Johnston", "Beach Boys"],
... [5, "Hank", "Marvin", "Shadows"],
... [6, "Bruce", "Welch", "Shadows"],
... [7, "Brian", "Bennett", "Shadows"],
... ]
>>> group_members = [Musician(*musician) for musician in musician_lists]
You populate group_members with Musician objects by transforming musician_lists with a list comprehension.
Note: You may have noticed that you used *musician to pass each list when creating class objects. The asterisk tells Python to unpack the list into its individual elements when creating the objects. In other words, the first object will have an .id attribute of 1, an .fname attribute of "Brian", and so on.
You now have a group_members list that contains seven Musician objects, four from the Beach Boys and three from the Shadows. You’ll next learn how you can use these with attrgetter().
Suppose you wanted to retrieve the .fname attribute from each group_members element:
>>> import operator
>>> get_fname = operator.attrgetter("fname")
>>> for person in group_members:
... print(get_fname(person))
...
Brian
Carl
Dennis
Bruce
Hank
Bruce
Brian
You first call attrgetter() and specify that its output will get the .fname attribute. The attrgetter() function then returns a function that will get you the .fname attribute of whatever object you pass to it. When you loop over your collection of Musician objects, get_fname() will return the .fname attributes.
The attrgetter() function also allows you to set up a function that can return several attributes at once. Suppose this time you want to return both the .id and .lname attributes of each object:
>>> get_id_lname = operator.attrgetter("id", "lname")
>>> for person in group_members:
... print(get_id_lname(person))
...
(1, "Wilson")
(2, "Wilson")
(3, "Wilson")
(4, "Johnston")
(5, "Marvin")
(6, "Welch")
(7, "Bennett")
This time, when you call attrgetter() and ask for both the .id and .lname attributes, you create a function capable of reading both attributes for any object. When run, your code returns both .id and .lname from the list of Musician objects passed to it. Of course, you can apply this function to any object, whether built in or custom, as long as the object contains both an .id and an .lname attribute.
Sorting and Searching Lists of Objects by Attribute With attrgetter()
The attrgetter() function also allows you to sort a collection of objects by their attributes. You’ll try this out by sorting the Musician objects in group_members by each object’s .id in reverse order.
First, make sure that you have access to group_members, as defined in the previous section of the tutorial. Then, you can use the power of attrgetter() to perform your custom sort:
>>> get_id = operator.attrgetter("id")
>>> for musician in sorted(group_members, key=get_id, reverse=True):
... print(musician)
...
Musician(id=7, fname='Brian', lname='Bennett', group='Shadows')
Musician(id=6, fname='Bruce', lname='Welch', group='Shadows')
Musician(id=5, fname='Hank', lname='Marvin', group='Shadows')
Musician(id=4, fname='Bruce', lname='Johnston', group='Beach Boys')
Musician(id=3, fname='Dennis', lname='Wilson', group='Beach Boys')
Musician(id=2, fname='Carl', lname='Wilson', group='Beach Boys')
Musician(id=1, fname='Brian', lname='Wilson', group='Beach Boys')
In this code snippet, you set up an attrgetter() function that can return an .id attribute. To sort the list in reverse by .id, you assign the get_id reference to the sorted() method’s key parameter and set reverse=True. When you print the Musician objects, the output shows that your sort has indeed worked.
If you want to show the object with the highest or lowest .id value, then you use the min() or max() function and pass it a reference to your get_id() function as its key:
>>> min(group_members, key=get_id)
Musician(id=1, fname='Brian', lname='Wilson', group='Beach Boys')
>>> max(group_members, key=get_id)
Musician(id=7, fname='Brian', lname='Bennett', group='Shadows')
You first create an attrgetter() function that can locate an .id attribute. Then you pass it into the min() and max() functions. In this case, your code returns the objects containing the lowest and highest .id attribute values. In this case, those are the objects with .id values of 1 and 7. You might like to experiment with this further by sorting on other attributes.
Calling Methods on Objects With methodcaller()
The final function that you’ll learn about is methodcaller(). It’s conceptually similar to attrgetter(), except it works on methods. To use it, you pass in the name of a method, along with any parameters that the method requires. It’ll return a function that will call the method on any object that you pass to it. The objects passed to methodcaller() don’t need to be of the same type. They only need to contain the method that you’re calling.
To learn about methodcaller(), you first need to enhance your existing Musician data class with a method:
>>> from dataclasses import dataclass
>>> @dataclass
... class Musician:
... id: int
... fname: str
... lname: str
... group: str
...
... def get_full_name(self, last_name_first=False):
... if last_name_first:
... return f"{self.lname}, {self.fname}"
... return f"{self.fname} {self.lname}"
...
You add a .get_full_name() method to Musician that accepts a single parameter named last_name_first with a default of False. This allows you to specify the order in which the names are returned.
Suppose you want to call .get_full_name() on each object in the previously defined group_members list:
>>> import operator
>>> first_last = operator.methodcaller("get_full_name")
>>> for person in group_members:
... print(first_last(person))
...
Brian Wilson
Carl Wilson
Dennis Wilson
Bruce Johnston
Hank Marvin
Bruce Welch
Here you use methodcaller() to create a function named first_last() that will call the .get_full_name() method of any object that you pass to it. Notice that you don’t pass any additional arguments to first_last(), so you receive back a list of the first names followed by the last names of all Musician objects.
If you want the first names to follow the last names, then you can pass in a True value for last_name_first:
>>> last_first = operator.methodcaller("get_full_name", True)
>>> for person in group_members:
... print(last_first(person))
...
Wilson, Brian
Wilson, Carl
Wilson, Dennis
Johnston, Bruce
Marvin, Hank
Welch, Bruce
Bennett, Brian
This time, you use methodcaller() to create a function named last_first() that will call the .get_full_name() method of any object passed to it, but it’ll also pass True to the last_name_first parameter. Now you receive a list of the last names then the first names of all the Musician objects.
Just like when you’re using attrgetter() to retrieve attributes, objects passed to methodcaller() can be either built in or custom. They only need to contain the method that you want to call.
FAQs
You’re now familiar with the purpose and uses of Python’s operator module. You’ve covered a lot of ground, and here, you’ll find a few questions and answers that sum up the most important concepts that you’ve covered in this tutorial.
You can use these questions to check your understanding or to recap and solidify what you’ve just learned. After each question, you’ll find a brief explanation hidden in a collapsible section. Click the Show/Hide toggle to reveal the answer. Time to dive in!
The operator module’s functions don’t replace Python’s built-in operators. Instead, you use many of the operator-equivalent operator functions in functional programming by passing them to higher-order functions. This isn’t possible with the built-in operators, because you can’t easily obtain a reference to them.
The operator module also provides the higher-order attrgetter(), itemgetter(), and methodcaller() functions. You can pass the functions that these three return to many common functions, including sorted(), max(), and min().
Unlike with built-in operators, you can serialize an operator module’s function, which is great because the pickle module is one of the most efficient ways of saving data. The downside is that it can’t usually cope with functions. You can, however, use it with operator module functions.
Python built-in operators will outperform both the operator module’s versions and lambdas, while the operator functions will outperform lambdas. If you make a single call to a lambda or the operator module equivalent, then you won’t notice much difference. However, when you make repeated calls, the difference becomes significant. The operator module’s functions aren’t replacements for the built-in operators when they’re all you need.
Using the operator functions can make your code more readable than the sometimes-confusing lambda function equivalents.
Do you have an interesting example of using the operator module? If you have a clever use case, why not share with your fellow programmers? Feel free to include it as a comment below.
Get Your Code: Click here to download the free sample code that shows you how to use Python’s operator module.
