Intern Objects

As we have seen in the previous chapter, Python creates an object if it doesn’t exist. And when the value repeats in the future, it just creates a new binding with the existing object.

However, in this video, you showed x and y both are 1500. Since 1500 exists outside the range of intern, a new obj was created, and x pointed to it. But why “y” didn’t get pointed even when it had the same value? This contradicts the previous chapter completely.

Please help.

@devavratk96 Let’s break it down to see what’s going on in more detail.

When you assign an expression to a Python variable, a few things happen:

1. The expression to the right of the assignment operator (=) is evaluated.
2. Your variable on the left side of the assignment operator is pointed to the object on the right. (It doesn’t matter what value it pointed to previously if any.)

When the expression is a value like a string literal ("Lorem ipsum"), a number (1500), or even a list, it will almost always result in allocating memory for a new object that will be created:

>>> x = "Lorem ipsum"
>>> y = "Lorem ipsum"
>>> x is y
False

Variables x and y refer to different memory locations, which are two distinct objects despite having the same value!

The only exception, in this case, would be interned objects, which are singletons. These only have one global instance in the whole application:

>>> x = "Lorem"
>>> y = "Lorem"
>>> x is y
True

The first assignment creates a brand new object in memory, but the second one reuses a “cached” string. It’s the same optimization technique as with small integers in Python, which may speed dictionary lookups in some cases.

When you assign another variable, you’re effectively aliasing some existing object, which will be referenced by more than one variable:

>>> x = "Lorem ipsum"
>>> y = x
>>> x is y
True

So, instead of creating a copy of the original string, now you have two ways of referencing the same object.

Things get more interesting when you try modifying the value of a variable. Depending on the type of the underlying object that the variable is pointing to, you’ll get different results. This touches upon the mutability of an object.

Mutable objects let you change their value without changing their identity. A Python list is an example of such an object:

>>> x = ["apple", "banana"]
>>> y = x
>>> x.append("orange")
>>> x is y
True
>>> x
['apple', 'banana', 'orange']
>>> y
['apple', 'banana', 'orange']

Aliasing allows you to update the list with variable x, and observe the modification with both variables.

This isn’t the case with immutable objects such as numbers:

>>> x = 1500
>>> y = x
>>> x is y
True
>>> x += 1  # Same as: x = x + 1
>>> x is y
False
>>> x, y
(1501, 1500)

When you modify a variable in place, you’re actually making a new assignment with an expression that includes a reference to the original variable. Because numbers in Python are immutable, addition creates a new object with the value of 1501, which is assigned back to the x variable. At the same time, variable y still points to the old value of 1500.

Thank you very much for the swift explanation. It is much clearer now. Also, as we saw in previous video, PyObject has Value, Type and Reference Count. When we have a mutable object such as a list, how does its system/diagram will look like? I assume the Value section will hold a reference to multiple value objects. Because, our main memory location of PyObject do not change even when we add or remove values. Am I correct?

@devavratk96 Python lists are special because they exhibit some useful features that would otherwise be associated with a few data structures in a low-level language like C.

On the one hand, Python lists are like traditional arrays because they provide so-called random access to their elements. In other words, if you know the index of an element, you can get the corresponding value immediately without any lookup because the index is basically the memory address. That effect can be achieved by laying out your elements as a sequence in one contiguous block of memory. However, it’ll only work as long as every element occupies precisely the same number of bytes. If so, you’ll be able to multiply that index by the size of a single element and offset it by the address of the first element in the sequence. One way of enforcing the same size for all elements is to keep only elements of the same type.

However, as you well know, lists in Python are heterogenous by letting you stuff in elements of arbitrary types:

elements = ["apple", 42, True, 3.14, 1 + 6j]

That’s possible with a linked list, which keeps track of elements scattered all over computer memory. Since elements no longer need to occupy a contiguous block of memory, each can have its own size. The tradeoff is a significantly slower lookup.

How is it possible that Python lists get the best of both worlds?

It seems they’re a hybrid of arrays and linked lists, which requires a little bit more memory. Python maintains a dynamic array of pointers to the actual elements. Every pointer, which represents some memory address, is a number that occupies a known number of bytes. Elements, on the other hand, can be anywhere and can have any size.

If you’re really curious about it, you can take a peek at the clever implementation in the CPython source code.

Hi,

I’m checking the case when a=280 and b=280. 280 is out of the [-5, 256] range but their ids are same in the output. Using Python 3.9. Has this range changed for newer version?

@karka Nothing’s changed. Here’s your example executed in Python 3.9:

Python 3.9.9 (main, Feb  3 2022, 09:45:44) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 280
>>> b = 280
>>> a is b
False
>>> id(a) == id(b)
False

How did you define those two variables? Python enables int interning for variables defined on the same line:

>>> a, b = 280, 280
>>> a is b
True
>>> id(a) == id(b)
True

I am using PyCharm.

a = 280
b = 280
print(a is b)

Output is True

@karka That’s strange because I’m unable to reproduce such a behavior.