Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Recapping Scopes and Closures
If you were holding your breath, you can let it out now! Congratulations on completing this deep dive into scopes and closures in Python. By using the debugger Thonny, you’ve walked through a code snippet to better understand how scopes and closures work.
In this Code Conversation video course, you:
- Refactored code to use more descriptive names
- Learned about local and nonlocal scopes and how functions access variables
- Understood how scopes open and close through inner and outer function calls
You used your interpreter to dig into Python and inspect dunder objects to find out how Python handles and stores variables.
For more informaton on the concepts covered in this video course, you can check out:
- Martijn Peters on Scopes and Closures: Stack Overflow
- Namespaces and Scope in Python
- Python Scope & the LEGB Rule: Resolving Names in Your Code
- Data Model: Python Documentation
Congratulations, you made it to the end of the course! What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment in the discussion section and let us know.
00:00 So this exploration into the dunder objects on your function objects in Python gives you a bit of a look under the hood of what’s going on when you’re defining scopes and closures and where Python makes the connections and stores variables, as well as the values of the variables.
00:19 I’ll start the recap with going again over the theoretical part of the deep-dive glossary. Now you might understand a little better what the different objects do because you have a bit of practical experience with them.
00:32 And then we’ll do another recap going over the different examples practically in the interpreter again. Let’s get started with the first object and its description.
00:43
The first one you got to know was the .__code__ object, which is present on all function objects, and it contains compiled function bytecode.
00:52 Now this is pseudo-compiled code that just contains the information of what does this function do so that you can execute it later on. This gets defined for every function object and gets executed when you call the function.
01:07
You can also execute the .__code__ object of a function explicitly.
01:13
Then you got to know two attributes on the .__code__ object, the first one being .co_cellvars, which is a tuple that has names of the different cell variables that can be referenced by a containing scope.
01:25
So this is from the perspective of the outer function, and it’ll contain variable names that inner functions could access from that specific function scope. And then somewhat related, you have .co_freevars, which is a tuple of names of free variables, which means that this is from the perspective of an inner function: what are the variables in a nonlocal scope that are accessible in this inner function? Then you also got to know the .__closure__ object, which also exists for each function object, but which will be None if the function does not contain another scope. And if it does, then it will be a tuple of cells that contains the bindings of the function’s free variables, which means the connections between the free variables that this function can access in the nonlocal scope and the pointers to their values.
02:25
Then you saw a Cell object, which represents these bindings for variables that can be referenced from different scopes.
02:34
And you also saw an attribute on the Cell object, which is .cell_contents. And that can be used to get the value of the cell and also to set it, which you both got to see in the walkthrough. Now, after this theoretical recap, let’s hop back into the interpreter and quickly walk over the different experiments again.
02:59
The outer() function keeps the references to any variables that are defined within its local scope in an object that is called .__code__, and then in the .co_cellvars argument, which is a tuple that keeps track of the variables that are defined within the local scope of this function and that might be accessible to an inner function. And an inner function does something similar, and inside of its .__code__ object on a different attribute, you get the variables that were defined in a nonlocal scope from the perspective of the inner() function. So in this case, the message variable was defined in the outer() function,
03:38
and this reference is stored inside of .__code__.co_freevars. And an inner function has a different dunder object called .__closure__ that contains cell objects, which represent the mappings between these variables, the variable names and the actual values.
03:59
And the values are then stored inside of this cell object. So in this case, I’m gonna have to step into the first one, because we only have one defined in this outer() function, and I don’t want reassign it, but actually just print it out.
04:12
Each cell object has an argument called .cell_contents, and this is where the actual value is stored. And then I also showed you that it can do funny things, like reassign this to a different string or to anything different really.
04:27 So I could assign it also to a number, right? This is just a value. And that this also reflects then to the output of that function object. So this is really where the value lives that was defined in a nonlocal scope and how it’s possible in Python that you still have access to this value, even after the scope that initially defined it has been dereferenced and doesn’t really exist anymore.
04:54
So I hope that this walkthrough through the code example from this Stack Overflow answer by Martijn Pieters that talks about closures and scopes in Python, and what is actually stored inside of .__closure__, was helpful to better understand this answer and try to give it another go now. In my opinion, the variable names used in here and function names aren’t ideal and make it a bit harder to understand, but maybe with this refactoring and then stepping through it with the interpreter, you get a better understanding of what’s happening there.
05:25 And I wanted to show you this also as an example of how you can approach understanding an answer or a question that might at first sight look a little difficult. You can always refactor, you can always use a debugger to step through it, and that usually helps to get a better understanding of what’s happening.
05:42 And now if you feel up for it, there is a second part to this answer.
05:47 So we’ve kind of gone up to here and talked about that. And then the second part to this answer, go ahead and read over that, try to refactor it and try to do this yourself and see if you can understand what’s happening here and why. I hope this walkthrough has been helpful for you.
06:04
Keep in mind that there’s links also in here that point to the Python docs about the data model, where you can learn more about .__closure__ and .__code__, and that we also have two relevant tutorials in Real Python: one about namespaces and scope in Python and another one about the LEGB rule specifically.
06:24 So make sure that you check out these tutorials as well if you’re interested in this topic, and if you want to learn more. Stay curious and keep digging and don’t get discouraged when sometimes you see a piece of code that’s a little hard to take apart and understand.
06:37 Just take your time and use the interpreter, use a debugger, and you can learn a lot on the way.
Become a Member to join the conversation.
Christopher Conlon on May 31, 2022
Interesting for those who are curious about how Python works under the hood. Unlikely to be much use to those who are only interested in practical Python application building (but could be).
It is sort of like the difference between working inside the space station and strapping on the EVA suit and taking a look at or working on the space station itself. If you are out there tinkering around with the solar panels then you are working on something completely different from running experiments inside. However, the inside work is dependent on that outside work.
Also, it is fun to step outside on an Extra Vehicular Activity.