Class Internals

This lesson seems like it suddenly becomes way too advanced, and I’m really struggling to understand the descriptor methods, and what their point or purpose is. Some intermediate step seems to have been skipped or not explained clearly.

Updating to my previous comment (I can’t see an edit button): after watching further, then going back and re-watching 2:00-4:00 several times, I can mostly understand how the PositiveInteger methods are working. What I’m still struggling with is the explanation between 4:00 and 5:00. I think this lesson needs to be reworked a bit.

Yep Ash,

As I attempted to say in the video, this isn’t easy stuff. Let me take one more stab at it.

Let’s start with two bits of terminology: first is the “class” which is the declaration, and second is the “object” which is an instance. When I write a method that belongs to a class, it is part of the declaration, whereas when I assign a value using “self”, it is adding that value to the object instance.

In the Ellipse example, the first use of “width” and “height” are part of the class declaration. So Ellipse gets two class attributes which themselves are instances of PostiveInteger objects.

An object instance has access to all of the class properties. Although “width” is declared at the class level, an instance of the Ellipse object can access it as well. If I had an instance of Ellipse called e, then e.width would be accessing “width” which was declared as a class attribute.

Inside __init__, that is what is happening. The line saying self.width = width is assigning the method’s “width” argument to self.width. In a normal case, that would overwrite the value contained in self.width. But, in this case, because self.width is an instance of a descriptor, instead of overwriting the value, the descriptors __set__ method is called.

The descriptor has implemented storing the value, so future access to self.width still gives you whatever was passed in as an argument to __init__, but it has side effects. The side effects check that the value being set is positive and raise ValueError if the validation fails.

Accessing self.width also invokes the descriptor. On access of self.width, instead of the object directly giving you the value, it uses the associated descriptor, calling its __get__ method. As the implementation of __get__ simply returns the stored value, the caller sees no difference in the behavior than if this wasn’t a descriptor.

Descriptors are tricky things and honestly you don’t come across them very frequently in Python. If you’re new to OO, you can accomplish a lot without fully groking descriptors, coming back to them later when you’ve had some practice.

I hope my second attempt at an explanation has made it better, rather than worse :)

Hi Christopher, thanks for writing such a thorough response. I think I’ve improved my understanding of this, and narrowed down where I think I’m still hung up. Here’s how I’m currently thinking:

The way that ‘width’ and ‘height’ are declared at the Ellipse class level and at the instance level gets tangled up in my understanding (from part 1) of class vs. instance attributes. Eg. declaring self.width in __init__ whilst there is also a ‘width’ class attribute, would surely result in an unrelated but identically named instance variable being created. I initially couldn’t see how you could create a new Ellipse instance - which results in __init__ setting the instance attributes width and height to the passed width and height arguments - when width and height are also class attributes (bear with me, I think I partly answer this further down). Here’s a simple non-descriptor example of what I’m getting at:

class Thing:
    size = 5
    def __init__(self, size):
        self.size = size

thing = Thing(10)

>>> Thing.size
5
>>> thing.size
10

In terms of line-by-line execution, class attribute ‘size’ is initialised as an int object with a value of 5, which any instances of Thing can access. Then later if/when a new Thing object is instantiated and __init__ is executed, its instance attribute ‘size’ is initialised to whatever ‘size’ argument was passed during instantiation, in this case 10. At that point, Thing.size and thing.size are coincidentally-named but totally separate attributes, right?

I then wonder: How does the Ellipse example not result in the class attributes (which are PositiveInteger objects) being overwritten by their same-named instance attribute counterparts (which are probably integers)? I think I figured out the answer:

In the Ellipse example, by the time that __init__ is called, due to the two class variables the interpreter already knows about PositiveInteger and its descriptoryness, so when anything (including __init__) tries to set self.size to something, the interpreter will call the descriptor’s __set__ method instead. But part of me still struggles with this.

What type is e.width - is it a PositiveInteger instance? (with an e.__dict__[width] attribute that’s an integer value 50?).

Hi Ash,

Your example is actually different from the descriptor example.

In your example, what is happening underneath when you assign self.width is there is no descriptor, so the instance variable overrides the class variable, and a new instance level attribute gets created. This is specific to the instance:

>>> class Thing:
...     width = 5
...
>>> Thing.width
5
>>> t = Thing()
>>> t.width
5
>>> t.width = 10
>>> t.width
10
>>> Thing.width
5
>>> u = Thing()
>>> u.width
5

The override of t.width only effects t, not changing Thing or u.

In the descriptor case, when you attempt to assign self.width the override hasn’t happened yet. The descriptor gets triggered instead. The mechanism in the language that decides what to do when you say self.width = X, is the same mechanism that triggers the descriptor. When there isn’t a descriptor, that mechanism overrides the value. When there is a descriptor, it calls the descriptor instead.

I’m not 100% sure how they’ve implemented it underneath, but you can almost think of it as there always being a descriptor, and the default descriptor’s behaviour is to overload the attribute in the instance.

Instance values actually are stored in a dictionary-like thing underneath, so you could think of it as a default descriptor’s job to replace the value in the dictionary.

To be clear, I haven’t played with the actual underlying C-code in Python, so it may not be built this way, but it isn’t a bad mental model.

Yep, that makes sense - your “In the descriptor case…” paragraph confirms where I’d started to get to, in terms of assumptions about how it must be working - so it’s great to get confirmation. It’s very rewarding to finally get it - thanks for your patience.

So this would mean that e.width is actually an int object, right?

Of course I had to go and confirm this, and found it interesting how deeply hooked-in to the language the descriptor protocol is - there’s no way around the descriptor’s __get__ method being called:

>>>type(e.width)
Getting width      # nice try
<class 'int'>

>>>e.width.__class__
Getting width      # but no cigar
<class 'int'>

>>>dir(e.width)
Getting width      # resistance is futile
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_count', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']