Checking for Membership in Generators

00:00 While many of the objects that you’re going to perform membership testing operations on are going to be containers of other elements, those are not the only types of objects in Python that you can use membership tests with.

00:14 The other type of objects are iterators, and generators are an example of iterators, which means that you can use membership test operations on them, but they’re a little bit different because generators don’t contain elements.

00:26 They generate them on demand. So when you use membership testing on a generator, you need to remember that doing a membership check actually consumes items while you’re checking them.

00:40 So let’s look at this real quick before continuing. Here, I have a generator defined. The code doesn’t really matter right now, just know this is a generator object.

00:48 You’ll see more about it in just a bit. But now for membership checking for a generator, this one produces numbers. So I’m going to ask is 2 in generator and it returns True.

01:00 What about 4 in generator, also True. Alright, so I’m going to tell you the secret that it goes up to the number four. So anything above that, it’s going to return False.

01:09 So let’s say 5 in gen is False. Alright, so but what about one? One is part of this generator. Now if I type 1 in gen, it also gives me False.

01:20 What about three? 3 in gen also False. So what’s going on here is does it skip numbers or what? Well, we know that two is in generator, right?

01:28 Which just, oh, look at that. Suddenly 2 in gen is also False. What about four? 4 in gen? It’s False as well. What’s going on?

01:38 This is different to what you’ve seen before, and it’s related to the fact that membership checking in a generator consumes the element. So at this point, the generator is completely exhausted and is only going to return False.

01:51 I’ll hop back to the slides for a moment and talk about this a little more.

01:56 So you can’t run it twice, right? That might bring some problems when you’re trying to first check for a membership in a generator and then want to do something later on.

02:04 Like you find out, okay, this item is in there, and then you want to run the generator again so that something happens with that item. That won’t work because you already consumed the generator in that case.

02:14 So that’s just something you need to be aware of. Another thing is that if you are dealing with an infinite generator, then if you’re checking for membership, it may just mean that it hangs if it never is going to find the item that you’re looking for.

02:29 Let’s look at the example where the generator consumes the item as it checks.

02:35 So I have a little generator function here, it’s called numbers_up_to, and then you’re passing in an upper bound and it just does a little bit of logging.

02:44 So it just gives us information of what’s happening, so it starts off by printing the string generating numbers, and then it loops over a range that you set up with an upper bound and then prints out it’s going to yield that specific number now, and then uses the yield keyword to give you access to that number, right?

03:03 Using yield here is what makes this function a generator function. I’m going to use this to show you what happens when you check for membership in a generator. For that now let me open up a Python REPL.

03:19 from generators import numbers_up_ to, and then I will create a generator that I’ll just call gen = numbers_up_

03:31 to(5).

03:35 So this is a generator object, as you can see when you print out its type or just pass it in like that. Alright, so this is of the class generator. Now you can still do membership checks on this and start off by saying, okay, is the number 2 in this generator,

03:53 and you’ll see that it returns True. So it finds it and it says yes, it’s in that generator. And you can also see the debugging printout that you’ve added to the numbers_up_to generator that starts off by saying, okay, generating numbers, and then it yields zero, it yields one, and then it yields two, which is the one that you were looking for so then it returns True. Alright, so let’s keep going.

04:14 What about the number three? Is 3 in the generator? And you can see again that it yields three and returns True, but you may notice that it’s not doing the same that the previous membership check did.

04:28 The previous membership check started off the generator and printed out the first logging message basically, and then you yielded 0, 1, 2, return True.

04:37 And so all of these numbers, they’re not part of the generator anymore. So now when you check for three, which is the next number, which just yields three and tells you True, which means that now if you would ask something like is 1 in the generator, you’ll see that the generator runs to the end and yields the final number in the generator, which goes from zero to four, and then it returns False because it’s at the end of the generator and it did not find 1.

05:02 Why? Because the generator yielded 1 already in your first membership check. So that first membership check consumed the item 1, so that later on you can’t find it anymore in that same generator.

05:14 And at this point, the generator is empty. So if you were to check for 4 in gen, for example, it’ll just return False. Now it’ll return False for any of the ones.

05:24 So also, if you run two again, right, the one that you ran first, you get False because the generator is exhausted and doesn’t give you any items anymore.

05:34 So this is something to be aware of. Membership checks work, but just like other operations on a generator, they consume the generator.

05:43 And this was just meant as a quick heads up, if you want to learn more about generators or if you’ve never heard of them before, then go ahead and check out the resources that we have on Real Python.

05:52 We have a tutorial and a video course on how to use generators and yield in Python.

05:57 And now let’s move on to custom classes.