Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Why Use the filter() Function?

Give Feedback

In this lesson, you’ll see why you’d want to use the filter() function rather than, for example, a for loop with an if statement. You’d get the same result, but functional programming allows you to chain function calls. This allows you to avoid side effects and have a line of code that gives you a quick snapshot of what is happening.

When doing functional programming, you have a bunch of functions as your basic building blocks, and then you can use and reuse them together in different contexts.

Comments & Discussion

matt7 on Feb. 20, 2020

You give some very good arguments why the filter style is good in this case. It would help if you could also have a quick discussion on the performance and memory usage impacts of this style of using filter vs. loops/if.

Lipsa on Feb. 28, 2020

where can i find the notebook for this series?

Ricky White RP Team on Feb. 28, 2020

Hi Lipsa. We don’t have notebooks for our series. Sorry.

sandeepranjan on March 21, 2020

I am getting an error when using the filter function:

mathS = tuple(filter(lambda x : x.field == 'math', scientists))
pprint(mathS)

Throws below error. I do have the necessary imports and rest of the code is identical to what is being shown in this tutorial -

Any Idea what am I missing ?

Traceback (most recent call last):
  File "/Users/sandeep/Library/Preferences/PyCharmCE2019.3/scratches/scratch.py", line 23, in <module>
    mathS = tuple(filter(lambda x : x.field == 'math', scientists))
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/fnmatch.py", line 51, in filter
    pat = os.path.normcase(pat)
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py", line 54, in normcase
    s = os.fspath(s)
TypeError: expected str, bytes or os.PathLike object, not tuple

Dan Bader RP Team on March 22, 2020

@sandeepranjan: It’s odd that you’re seeing an exception somewhere in the os module (posixpath.py and fnmatch.py in the Traceback)… I don’t see anything wrong with your filter expression upon first inspection.

I’m wondering if this is an issue with your local Python environment, maybe you could try your program on a different machine or inside a Docker container with a clean CPython install to see if the error goes away.

jaman on March 23, 2020

Hi Dan, thanks for the great videos. I want to understand your thought process about when/if data should be stored in a pandas dataframe instead of storing data in basic types like tuples, lists, dictionaries, etc. My impression is that data analysis/viz/modeling work uses pandas, but I’m wondering what you consider while deciding if data should belong in a pandas df.

Dan Bader RP Team on March 23, 2020

IMO the main benefits of using Pandas (which is built on top of NumPy) over a homegrown solution are:

  1. Ease of use and access to common functionality (eg. loading & reshaping data)
  2. Performance benefits due to vectorization

Nanogines on March 27, 2020

What editor do you use?

Rashikraj Shrestha on March 28, 2020

# we can reuse filter function with a reusable function
def nobel_filter(x):
    return x.nobel is True
tuple(filter(lambda x: nobel_filter, Scientists))
(Scientist(name='Ada Lovelace', field='math', born=1815, nobel=False),
 Scientist(name='Emmy Noether', field='math', born=1882, nobel=False),
 Scientist(name='Marie Curie', field='math', born=1867, nobel=True),
 Scientist(name='Tu Youyou', field='physics', born=1930, nobel=True),
 Scientist(name='Ada Yonath', field='chemistry', born=1939, nobel=True),
 Scientist(name='Vera Rubin', field='chemistry', born=1928, nobel=False),
 Scientist(name='Sally Ride', field='physics', born=1951, nobel=False))

what is the issue over here?? i could not exploit it

Sagar Rathod on March 30, 2020

Hi @Rashikraj Shrestha, You should have called that function nobel_filter(x) inside lambda insted of just nobel_filter.

Thanks Dan, for Nice Explanation.

Rashikraj Shrestha on March 31, 2020

@Sagar Rathod, but in this tutorial it is clearly instructing to use nobel_filter only

Victor R Cardoso on March 31, 2020

@Rashikaj, you added the extra lambda expression:

tuple(filter(lambda x: nobel_filter, Scientists))

This function will return everything as True, given that for each case lambda x will return the function nobel_filter (without calling it!).

You’ll need to remove the lambda to use the function:

tuple(filter(nobel_filter, Scientists))

In this case it will call the function for each case.

Otherwise, if you want to keep the lambda you’ll need to make it calling the x case explicitly:

tuple(filter(lambda x: nobel_filter(x), Scientists))

I have not checked, but this should work fine.

Thank you for the discussion.

Dan Bader RP Team on April 1, 2020

Yes that’s correct, good explanation Victor :)

Become a Member to join the conversation.