Inspecting Dunder Objects

00:00 In this lesson, you’ll dive a little behind the scenes of what happened here and see where Python stores the different variables that you’ve been working with and how it keeps track of the nonlocal scope of a function even after the scope of the function was already dereferenced.

00:17 And for this, I’m going use an iPython interpreter session instead of continuing with Thonny here. You could also continue down here in the debugger so you still have access to anything that you wrote in here, and you could work with it in the interpreter session as well, but I’m just going to move over to an iPython session in my terminal because it gives me a little more space and also a somewhat better autocomplete and syntax highlighting.

00:46 So if I paste the exact same code into the iPython interpreter session here and press Enter, you again see the output that you got and you remember how it got to be by stepping through it before. And now we are going to inspect a little more this inner_returned object, as well as the outer function object.

01:08 Before you’ll go on this dive with me down below the surface of Python, here’s a chance to take a quick look at a couple of the vocabulary that I’ll be using.

01:17 Now don’t get overwhelmed when this feels like a lot. We’ll recap it later on again, and some of it might make sense while you hear me walk through it, and some of it might not make sense, and that’s okay as well.

01:29 I would suggest to just go with it for now, then check whether you understood it when we are going in the recap and otherwise just rewatch it a second time, check out the docs, there will be some links around, and just approach it with a relaxed mindset.

01:42 You don’t need to understand everything to get the point of something. So with that said, here’s some of the vocabulary that you will hear. You will hear about a .__code__ object, and you will hear about two attributes on this .__code__ object: first, .co_cellvars, and then also .co_freevars.

02:02 And then you will hear about another object on the function object, which is .__closure__. Further, you’ll hear about a Cell object and an attribute on the Cell object that’s called .cell_contents.

02:19 Now you can pause these screens and read what they do or go to the documentation if you prefer to do this before the walkthrough, but my suggestion is to just dive in, hold your breath—or better, don’t hold your breath because I think it’s going to be a bit too long for that—but you’ll come out on the other side being just a little bit wiser.

02:39 So first of all, I’d be curious to see where did this message variable get stored? And there’s two parts to this. So, first of all, when you defined the outer() function, then it created

02:51 a .__code__ object that has a couple of arguments to it, and the one that’s relevant for us at the moment is the .co_cellvars,

03:01 because this is where an outer function stores the variables that get defined in there that might be accessed by an inner() function. So this is where the outer() function stored message.

03:14 And there could be more of those. This is a tuple, and if there were other variables defined in here, you would see them come up here in the tuple. But in this example, we only have this one message variable.

03:25 This is where it gets defined in the outer() function, and then you also have the inverse thing for the inner() function. So let’s check out the inner_returned function object, and it has this .__closure__,

03:42 which let’s look at it first before going deeper.

03:47 You see this is another tuple and it holds, in this case, just one value, and that is a cell value. Yeah, again, there’s one value in here because we only have one variable defined, but if there were more, then you would just see this tuple continued with additional cell objects in there.

04:05 Let’s look at how you can get the value of this. If you look at this first cell object, which is going to hold the reference that connects the message variable with a value, then you can find it by going—so, first of all, you have to access this first item in the tuple, which is this cell object—and then you see in the autocomplete already, it has the argument called .cell_contents, and this is what holds the actual value, so this is where message relates to the string 'world'.

04:35 This is where it got defined and where it’s stored inside of the .__closure__ of the inner_returned function object.

04:44 Now the inverse to—like the reference to the variable name also sits somewhere, and that sits inside of something called .__code__, just like the one that you looked at up here for the outer() function.

04:58 The outer() function has co_cellvars to hold this reference, and the inner one, .co_cellvars is empty, but instead the .co_freevars holds this reference.

05:12 So the name of the variable that is accessible in the inner scope of the inner() function, and this is how this connection gets made. So when you define the outer() function, then it puts any variables that are in there into this .__code__.co_cellvars and gives the name of them in here.

05:30 And these are variables that might be accessible in the scope of an inner function. And then the inner function that you define inside of an outer function also has this .__code__ object and has .co_freevars, which makes the connection back to these are variables that are defined in a nonlocal scope from the perspective of this function, but they are still accessible.

05:53 And then if you’re actually looking for the content, like what does this binding point to, which value, then you have to go into .__closure__. So again, let’s look at this .__closure__. Again, print it out.

06:10 .__closure__ is a cell object.

06:14 You are looking at the first, and in this case only, cell object in here, which is the message variable that you saw connected up here, right?

06:23 And this points to the value that is stored in .cell_contents, which is 'world' in that case. And this is why you can go ahead and keep calling inner_returned(), and you keep getting back this value, world, here, which is because the function definition looks like that, that you were just printing out this message, and you can keep doing that.

06:49 Okay. Keep saying inner_returned() and it’ll always call it, it’ll always return world—until I do something, which you probably shouldn’t do, because anytime that you’re dealing with these dunder objects in Python, this is usually something that’s for internal use of Python, and that’s not really something as an end user you’re meant to be messing with, but you can. Python gives you often these opportunities to.

07:15 So you can do something fun, like going into inner_returned.__closure__[0].cell_contents. And instead of just printing it, you’re going to now assign it to a new value. You’re going to say "oops".

07:29 And that changed the value of this variable that you before defined in the nonlocal scope of the inner() function, so in the local scope of the outer() function, by the variable name message, but now you just stepped in there.

07:42 You went into the function object, into its .__closure__, into the cell object that is stored in there, which is this reference between the variable name and its value, and then you change the value of this variable name.

07:54 So the variable name is still message, but now if you go ahead and say inner_returned(), you get out oops instead of world, which you had before, but the function signature is still the same.

08:05 So if you look at—see if that works—inner_returned,

08:12 you can see that here you have the __code__ object. You can keep stepping in there, inner_returned.__code__. See what you get here. And then you have the co_freevars, which you’re looking for in that case—

08:30 .co_freevars, and this is still going to point to message, right? Well, it’ll be done when I get the dir() here. So let me just print out the actual value of this.

08:41 I can see that it’s still this tuple with a variable of the name message, but now the value of the mapping of this message variable, you changed it to point to oops, instead of world, which is what it was when you defined it inside of the outer() function.