What Is the filter() Function?

Functional Programming in Python Dan Bader 03:17

In this lesson, you’ll learn that filter() is one of the functional programming primitives (or building blocks) available in Python and that it’s useful in a number of contexts.

You’ll see how you can use filter() to play with your data set. You’ll write a filter expression to see which items in your data set meet particular criteria. filter() takes another function object, and you can define a function object inline with lambda expressions.

00:00 The filter() function is one of the functional programming primitives that you can use in your Python programs. It’s built into Python. Here, I’m on Python 3. This would also work on Python 2.

00:13 The filter() function is built-in and it has maybe a slightly complicated docstring. It says here, the filter() function returns […] an iterator yielding those items of iterable for which function(item) is true. If function is None, return the items that are true. Okay. What does that mean?

00:32 Right? That’s pretty intense. We’re just going to play with it and then you can see how we can use the filter() function to play with this data set and to apply some filtering.

00:41 What I want to do now is I want to write a filter() function—a filter() expression that gives me all of the scientists in this list, or actually, gives me a new list of scientists that have won the Nobel Prize. Or, the opposite, right?

00:55 Like, we want to filter this data set by whether or not that person won the Nobel Prize at some point. Okay, let’s take this step by step. All right? I’ve got the filter() function here.

01:05 It looks like this takes another function object, and if you want to define a function object inline, you can use the built-in lambda expression.

01:15 Here, I’m going to say lambda x, so, this is the object that will be passed in. So lambda of x, I want this to return True or False depending on whether or not the nobel flag is set to True or False.

01:28 I can go here and I can say x.nobel is True. So, if you’re wondering how a lambda works, it’s basically a one-line function, like a shortcut for defining a function in Python. So, you can put arguments, and then you have one expression that gets evaluated.

01:46 You don’t have to do return or anything—that’s automatic. And it will just evaluate this expression and return it back out of the lambda function. In this case, I’m just comparing x.nobel to True.

01:58 I could have shortened that and just said x.nobel, but honestly, I think this is nicer because it actually makes sure we’re returning a boolean and I think that’s the better way to do it. In this case, I would say—all right, we’re checking here and we’re passing in an iterable, which is the list of scientists here. Okay.

02:21 What do you think is going to happen when I run this? Oh, okay! So, depending on whether you’re running this on Python 2 or 3, you’re going to see a different result here. On Python 2, this will have created a list object containing all of these filtered scientists, or the scientists that passed this test here, so this is like a test function.

02:41 And on Python 3, this actually returns a filter object, which is an iterable. It works like an iterator, so we need to extract the next item from it.

02:51 And well, actually, why don’t we do that?

02:57 Let’s call this fs for filtered scientists, and then we can just say next(fs) and that’s going to give me all of these filtered scientists. And you can see here, this only contains scientists that had .nobel set to True, and now we’re hitting a StopIteration exception, and we’re at the end of that.

brunofl on July 14, 2019

Nice work! how can I have a terminal or idle shell showing command help info like you did in these videos?

Dan Bader RP Team on July 15, 2019

Thanks @brunofl! I’m using an alternative Python REPL called bpython in my videos. You can learn more about it here: bpython-interpreter.org/

brunofl on July 17, 2019

Nice, I will give it a try. Thanks!

vtzast on April 1, 2020

Alternatively, in Python 3.x instead of using the next() we could use a list

list(filter(lambda x: x.nobel is True, scientists))

or a tuple

tuple(filter(lambda x: x.nobel is True, scientists))

kiran on Aug. 18, 2020

How can i print all values in namedtuple. i am trying like this

>>> pprint(Scientist)
<class '__main__.Scientist'

It return Scientist object.

Dan Bader RP Team on Aug. 18, 2020

@kiran: You’re printing the Scientist class. To get a nice string representation of a Scientist instance you need to create a Scientist object first:

import collections
import pprint

Scientist = collections.namedtuple('Scientist', [
    'name',
    'field',
    'born',
    'nobel',
])

obj = Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False)
pprint.pprint(obj)

Output:

Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False)

Check out our “Object-Oriented Programming (OOP) With Python” Learning Path to learn more about OOP in Python and the difference between classes and instances/objects.

kiran on Aug. 18, 2020

Hi @Dan Bader ya for one obj you can print like this, but what about if i have more then one object.? (you printed all Scientist information in single object)

Dan Bader RP Team on Aug. 19, 2020

Maybe I’m misunderstanding you, but it works just the same with multiple objects:

>>> objs = [Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False), Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False), Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False)]
>>> print(objs)
[Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False), Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False), Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False)]
>>> pprint.pprint(objs)
[Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False),
 Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False),
 Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False)]

kiran on Aug. 19, 2020

@Dan Bader Thanks for clarification.

PrasadChaskar on Dec. 10, 2020

If we iterate filter object through for loop it does not print fist Scientist name if scientist has nobel value True.

scientists = (
    Scientist(name='C.V. Raman', field='Physics ', nobel=True),
    Scientist(name='Ada Loverace', field='Math', nobel=False),
    Scientist(name='RavindarnathTogore', field='Literature', nobel=True),
    Scientist(name='APJ Abul Kalam', field='Aerospace ', nobel=False),
)
nobel_winners = filter(lambda x: x.nobel is True, scientists)
for winner in nobel_winners:
    print(winner)

Why did this happen?

Bartosz Zaczyński RP Team on Dec. 10, 2020

@PrasadChaskar What do you mean? It works for me:

>>> from collections import namedtuple
>>> Scientist = namedtuple("Scientist", "name field nobel")
>>> scientists = (
...     Scientist(name='C.V. Raman', field='Physics ', nobel=True),
...     Scientist(name='Ada Loverace', field='Math', nobel=False),
...     Scientist(name='RavindarnathTogore', field='Literature', nobel=True),
...     Scientist(name='APJ Abul Kalam', field='Aerospace ', nobel=False),
... )
>>> nobel_winners = filter(lambda x: x.nobel, scientists)
>>> for winner in nobel_winners:
...     print(winner)
... 
Scientist(name='C.V. Raman', field='Physics ', nobel=True)
Scientist(name='RavindarnathTogore', field='Literature', nobel=True)

PrasadChaskar on Dec. 10, 2020

@Bartosz Zaczyński When I run first time it was not shown but now its working fine. Thanks for Clarification

sreesankar15207 on May 15, 2021

Hello. You wrote a lambda expression lambda x : x.nobel is True. The is word checks whether a variable is pointing to a particular memory.So how can you do that?

Dan Bader RP Team on May 15, 2021

Ah yes, good point! I could’ve simplified

lambda x: x.nobel is True

lambda x: x.nobel

Because all we wanted to know for the filtering is whether the .nobel field is “truthy” or not. But to make that check more explicit, I went with the x.nobel is True comparison.

It’s a standard convention to use is with the built-in constants True, False and None (more context here).

sreesankar15207 on May 16, 2021

Thank you for clarification sir.

Become a Member to join the conversation.