Single and Double Underscores in Python Names

Public Interfaces and Naming Conventions in Python

As a Python programmer, you’ll frequently work with public interfaces, or application programming interfaces (API). An API is a type of programming interface that offers a service to other parts of a program or other programs.

For example, the Python standard library has many modules and packages that provide certain services. To use these modules and packages, you need to access their public components, such as classes, functions, variables, constants, and modules. All these objects are part of the module or package’s public interface. They’re available for you to use directly in your code.

However, many of these packages and modules define objects that aren’t intended for direct access. These objects are for the internal use of the specific package or module only. They’re not part of its public interface.

In the context of object-oriented programming, languages like C++ and Java have the notion of public and private methods and attributes. In these languages, you can use these types of class members as follows:

• Public: You can use them in your own code or client code.
• Private: You can use them only from inside the defining class and its subclasses.

These languages have specific keywords and syntax to define public and private members in their classes. Once you declare a member as private, you can’t use it outside the class because the language restricts access. So, private members aren’t part of the class’s public interface, and there’s no way to access them.

In contrast, Python doesn’t have the notion of public and private members. It has neither dedicated keywords nor syntax for defining them. So, you can always access the members of a Python class.

If Python doesn’t have a specific syntax to define when an object is part of a public interface, then how do you tell your users that they can or can’t use a given class, method, function, variable, constant, or even module in their code?

To approach this question, the Python community has well-established naming conventions. You should observe these naming conventions to explicitly indicate whether other developers should directly use your variables, constants, functions, methods, and modules in external code.

Note that the naming conventions don’t restrict access to objects. They’re just a warning to other developers using your code. Because of this, Python doesn’t use the terms public and private. Instead, it uses the terms public and non-public.

In Python, if a name starts with a letter in uppercase or lowercase, then you should consider that name public and, therefore, part of the code’s API. In contrast, if a name starts with an underscore character (_), then you should consider that name non-public, meaning it’s not a part of the code’s API.

The Python community uses the underscore character (_) as part of other naming conventions. Here’s a summary of what PEP 8 says about using this character in names:

Note that only two of these conventions enforce specific Python behaviors. Using double leading underscores triggers name mangling in Python classes. You’ll learn more about this behavior in the section on name mangling.

Additionally, those names with double leading and trailing underscores that are listed in the Python data model trigger internal behaviors in specific contexts. You’ll also learn more about this topic in the section on dunder names in Python.

In the following section, you’ll learn more about using a single leading underscore when naming your Python objects.

Single Leading Underscore in Python Names

As you already learned, Python doesn’t have dedicated keywords and syntax to make objects public or private. Therefore, Python calls its objects public or non-public and doesn’t restrict access to them.

>>> public = "This is a public name."
>>> public
'This is a public name.'

>>> _non_public = "This is a non-public name."
>>> _non_public
'This is a non-public name.'

Public and Non-Public Names in Modules and Packages

Python modules might be one of those places where you’ll need to decide which names should be public and which should be non-public. At the module level, you can have some of the following objects:

• Constants
• Variables
• Functions
• Classes

You may need to allow your users to access these objects as part of your module’s API. However, in some cases, you may need to communicate that some of your module-level objects aren’t intended to be part of the API.

Some common examples of module-level non-public objects include:

• Internal variables and constants
• Helper functions and classes

In the following sections, you’ll learn about these types of objects. You’ll also learn about non-public modules. Yes, you can even have non-public modules in your packages.

_count = 0

def increment():
    global _count
    _count += 1

def decrement():
    global _count
    _count -= 1

def get_count():
    return _count

_PI = 3.14

class Circle:
    def __init__(self, radius):
        self.radius = radius

    def calculate_area(self):
        return round(_PI * self.radius**2, 2)

_PI = 3.14

class Circle:
    def __init__(self, radius):
        self.radius = _validate(radius)

    def calculate_area(self):
        return round(_PI * self.radius**2, 2)

class Square:
    def __init__(self, side):
        self.side = _validate(side)

    def calculate_area(self):
        return round(self.side**2, 2)

def _validate(value):
        if not isinstance(value, int | float) or value <= 0:
            raise ValueError("positive number expected")
        return value

Wildcard Imports and Non-Public Names

Sometimes, you’ll find Python libraries that use wildcard imports of the form from module import * to bring all the names defined in a module into your current namespace. The Tkinter library for GUI development is a good example of this practice. These types of imports aren’t particularly common because Python’s style guide, PEP 8, discourages their use:

Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. (Source)

However, sometimes wildcard imports are useful and quick. To use wildcard imports properly, you should know that they don’t import non-public names. To confirm this Python behavior, open a REPL session in the directory where your shapes.py file lives. Then run the following code:

>>> from shapes import *

>>> dir()
[
    'Circle',
    'Square',
    ...
]

In this code snippet, you can see that your wildcard import only imports the Circle and Square classes. The _PI constant and the _validate() helper function aren’t in your current namespace.

Python enforces the above behavior. However, this behavior only applies to wildcard imports. You can still access non-public names using other import statement forms:

>>> from shapes import _PI

>>> _PI
3.14

>>> import shapes

>>> shapes._PI
3.14

Using an import form other than a wildcard import, you can bring any non-public name to your current namespace. In this example, you import the _PI constant from shapes.py even if it’s a non-public object. You can also access _PI by using dot notation on the shapes module. Keep in mind that doing something like this bypasses the intention of the established naming convention in regard to non-public names.

Class With Non-Public Members

As you already learned, Python doesn’t enforce a strict distinction between public and private names. Instead, it has an established naming convention to differentiate between public and non-public names. However, it’s up to you as a developer to honor that convention. All Python names are accessible in one way or another unless they live in a different scope.

The distinction between public and non-public names is even more relevant in the context of object-oriented programming. Here’s how PEP 8 defines these terms when referring to the attributes of a class:

Public attributes are those that you expect unrelated clients of your class to use, with your commitment to avoid backwards incompatible changes. Non-public attributes are those that are not intended to be used by third parties; you make no guarantees that non-public attributes won’t change or even be removed. (Source)

Again, public class attributes and methods—also known as members—are part of the class’s public interface, while non-public members aren’t part of it. It’d be best not to use non-public members outside of their defining class. Non-public members are non-public for a reason, so you should respect that.

In most cases, non-public members exist only to support the internal implementation of a given class and may disappear at any time, so your users shouldn’t rely on them.

In the following sections, you’ll explore some use cases of non-public attributes and methods in Python classes.

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

>>> from point import Point

>>> point = Point(12, 5)

>>> # Access the coordinates
>>> point.x
12
>>> point.y
5

>>> # Update a coordinate
>>> point.x = 24
>>> point.x
24

>>> # Assign an invalid data type
>>> point.y = "10"
>>> point.y
'10'

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @property
    def x(self):
        return self._x

    @x.setter
    def x(self, value):
        self._x = _validate(value)

    @property
    def y(self):
        return self._y

    @y.setter
    def y(self, value):
        self._y = _validate(value)

def _validate(value):
    if not isinstance(value, int | float):
        raise ValueError("number expected")
    return value

>>> from point import Point

>>> point = Point(12, 5)

>>> point.x
12
>>> point.y
5

>>> point.x = 24
>>> point.x
24

>>> point.y = "10"
Traceback (most recent call last):
    ...
ValueError: number expected

>>> point._y
5
>>> point._x
24

>>> point._y = "10"
>>> point._y
'10'
>>> point.y
'10'

import csv

class CSVFileManager:
    def __init__(self, file_path):
        self.file_path = file_path

    def read_csv(self):
        return self._read(delimiter=",")

    def read_tsv(self):
        return self._read(delimiter="\t")

    def _read(self, delimiter):
        with open(self.file_path, mode="r") as file:
            return [row for row in csv.reader(file, delimiter=delimiter)]

Double Leading Underscore in Classes: Python’s Name Mangling

In the context of Python classes, you’ll find another naming convention that uses two leading underscores in attribute and method names. Unlike a single leading underscore, which doesn’t produce any Python response, double leading underscores trigger a behavior that’s known as name mangling in Python’s lingo.

In the following sections, you’ll learn what name mangling is, how it works, and what it aims to do in Python.

>>> class SampleClass:
...     def __init__(self, attribute):
...         self.__attribute = attribute
...     def __method(self):
...         print(self.__attribute)
...

>>> sample_instance = SampleClass("Hello!")
>>> vars(sample_instance)
{'_SampleClass__attribute': 'Hello!'}

>>> sample_instance.__attribute
Traceback (most recent call last):
    ...
AttributeError: 'SampleClass' object has no attribute '__attribute'

>>> sample_instance.__method()
Traceback (most recent call last):
    ...
AttributeError: 'SampleClass' object has no attribute '__method'

>>> sample_instance._SampleClass__attribute
'Hello!'

>>> sample_instance._SampleClass__method()
Hello!

class A:
    def __init__(self):
        self.__attr = 0

    def __method(self):
        print("A.__attr", self.__attr)

class B(A):
    def __init__(self):
        super().__init__()
        self.__attr = 1  # Doesn't override A.__attr

    def __method(self):  # Doesn't override A.__method()
        print("B.__attr", self.__attr)

Trailing Underscores in Python Names

Another naming convention that you’ll find in Python code is to use a single trailing underscore. This convention comes in handy when you want to use a name that clashes with a Python keyword or a built-in name. In this situation, adding a trailing underscore will help you avoid the issue.

For example, sometimes when you’re starting to learn Python, and you’re diving into using the built-in list data type, you may be tempted to do something like the following:

>>> list = [1, 2, 3, 4]
>>> list
[1, 2, 3, 4]

Even though this code works, it clashes with the list name, which is reserved for a built-in type. If you continue using the same interactive session, then you’ll be surprised when you get an error like the following:

>>> list("Pythonista")
Traceback (most recent call last):
    ...
TypeError: 'list' object is not callable

What just happened? Why can’t you call the list() constructor? Your previous code overrode the built-in name, and now the name list points to a list object rather than to the list class.

You can fix this issue by appending an underscore to the clashing name. Note that for the code below to work, you need to restart your interactive session or use the del statement to remove your custom name from your current namespace:

>>> list_ = [1, 2, 3, 4]
>>> list_
[1, 2, 3, 4]

>>> list("Pythonista")
['P', 'y', 't', 'h', 'o', 'n', 'i', 's', 't', 'a']

In this new version of your code, your name doesn’t override the built-in name. So, the list() constructor works as expected.

Another situation where you’d need to use this trailing underscore naming is when you need to use a name that clashes with a keyword. Consider the following toy example:

class Passenger:
    def __init__(self, name, class_, seat):
        self.name = name
        self.class_ = class_
        self.seat = seat

    # Implementation...

In this code snippet, you use a trailing underscore to define the passenger’s class. If you don’t do it like this, then you’ll get an error because class is a Python keyword, and you can’t use it with a different purpose in your code.

Even though the trailing underscore trick works, it’s probably best to use a synonym or a multiword name. In the above example, your code will probably look better if you use something like reserved_class to name the argument and the corresponding attribute.

Dunder Names in Python

In Python, names with double leading and trailing underscores (__) have special meaning to the language itself. These names are known as dunder names, and dunder is short for double underscore. In most cases, Python uses dunder names for methods that support a given functionality in the language’s data model.

Dunder methods are also known as special methods, and in some informal circles, they’re called magic methods. Why magic? Because Python calls them automatically in response to specific actions. For example, when you call the built-in len() function with a list object as an argument, Python calls list.__len__() under the hood to retrieve the list’s length.

In general, dunder names are reserved for supporting internal Python behaviors. So, you should avoid inventing such names. Instead, you should only use documented dunder names. In the end, creating a custom dunder name won’t have a practical effect because Python only calls those special methods that the language defines.

A few examples of commonly used dunder methods include:

These are just a sample of all the special methods that Python defines. As you can conclude from their descriptions, all these methods support specific Python features. You can provide implementations for these methods in your custom classes so that they support the related features.

To illustrate, say that you want to create a ShoppingCart class to manage the cart in an online shop. You need this class to support the len() function, which should return the number of items in the cart. Here’s how you can provide this support:

class ShoppingCart:
    def __init__(self):
        self.products = []

    def add_product(self, product):
        self.products.append(product)

    def get_products(self):
        return self.products

    def __len__(self):
        return len(self.products)

Your class keeps track of the added products using a list object. Then, you implement methods for adding a new product to the cart and retrieving the list of current products. The .__len__() special method returns the number of items in .products using the built-in len() function.

By implementing .__len__(), you ensure that your ShoppingCart class also support the len() function:

>>> from cart import ShoppingCart

>>> cart = ShoppingCart()
>>> cart.add_product("keyboard")
>>> cart.add_product("mouse")
>>> cart.add_product("monitor")

>>> len(cart)
3

In this snippet, you create a shopping cart and add three items. When you call len() with the cart instance as an argument, you get the number of items that are currently in the cart.

Finally, Python also has some dunder names that don’t refer to special methods but to special attributes and variables. A couple of the most commonly used ones are the following:

• __name__ uniquely identifies the module in the import system.
• __file__ indicates the path to the file from which a module was loaded.

You’ll often find __name__ in a common Python idiom that’s closely related to executable scripts. So, in many executable Python files, you’ll see a code snippet that looks like the following:

# ...

def main():
    # Implemention...

if __name__ == "__main__":
    main()

The name-main idiom allows you to execute code when you run the containing file as a script but not when you import it as a module.

Other Usages of Underscores in Python

Up to this point, you’ve learned about some Python naming conventions involving either leading or trailing underscores. You’ve learned about public vs non-public names, name mangling, and dunder names.

In this section, you’ll quickly explore other use cases of underscores in Python names. Some of these use cases include the following:

• Placeholder variable in REPL sessions
• Throwaway variables in loops and other constructs
• Wildcards in structural pattern matching
• Named tuple methods

In the context of a REPL session, the underscore character has an implicit role. It works as a special variable containing the result of the last evaluated expression:

>>> 12 + 30
42
>>> _
42

>>> pow(4, 2)
16
>>> _
16

In these examples, you evaluate two different expressions. Expressions always evaluate to a concrete value, which Python automatically assigns to the _ variable after the evaluation. You can access and use the _ variable as you’d use any other variable:

>>> numbers = [1, 2, 3, 4]

>>> len(numbers)
4

>>> sum(numbers) / _
2.5

In this example, you first create a list of numbers. Then, you call len() to get the number of values in the list. Python automatically stores this value in the _ variable. Finally, you use _ to compute the average of your list of values.

Throwaway variables are another common use case of underscores in Python names. You’ll often see them in for loops and comprehensions where you don’t need to use the loop variable in any computation.

To illustrate, say that you want to build a list of lists to represent a five-by-five matrix. Every row will contain integer numbers from 0 to 4. In this situation, you can use the following list comprehension:

>>> matrix = [[number for number in range(5)] for _ in range(5)]

>>> matrix
[
    [0, 1, 2, 3, 4],
    [0, 1, 2, 3, 4],
    [0, 1, 2, 3, 4],
    [0, 1, 2, 3, 4],
    [0, 1, 2, 3, 4]
]

In this example, the outer list comprehension, [... for _ in range(5)], creates five lists. Each list represents a row in the resulting matrix. Note how you’ve used an underscore (_) as the loop variable. You do this because you don’t need this variable in the inner list comprehension, [number for number in range(5)], which fills each row with values.

Structural pattern matching was introduced to Python in version 3.10. It uses a match … case construct to compare an object to several different cases. Such statements are effective at deconstructing data structures and picking out individual elements. Python treats a single underscore as a wildcard in a case statement, so it’ll match anything. You often use it to alert the user that an object doesn’t have the expected structure.

The following example shows a recursive function that uses pattern matching to sum a list of numbers:

>>> def sum_list(numbers):
...     match numbers:
...         case []:
...             return 0
...         case [int(first) | float(first), *rest]:
...             return first + sum_list(rest)
...         case _:
...             raise ValueError(f"can only sum lists of numbers")
...

The last case statement uses the _ wildcard and will match if numbers isn’t a list with integer and floating point numbers. You can try it in action:

>>> sum_list([1, 2, 3])
6

>>> sum_list(["x", "y"])
Traceback (most recent call last):
    ...
ValueError: can only sum lists of numbers

When you try to sum a list of strings, then your final case triggers and raises the ValueError.

Finally, have you used named tuples before? They can help you improve the readability of your code by providing named fields to access their items using dot notation. One odd characteristic of named tuples is that some of the attributes and methods in their pubic interface have a leading underscore in their names:

>>> from collections import namedtuple

>>> Point = namedtuple("Point", "x y")

>>> point = Point(2, 4)
>>> point
Point(x=2, y=4)

>>> dir(point)
[
    ...
    '_asdict',
    '_field_defaults',
    '_fields',
    '_make',
    '_replace',
    'count',
    'index',
    'x',
    'y'
]

Besides the .count() and .index() methods, which named tuples inherit from tuple, named tuples provide three additional methods: ._asdict(), ._make(), and ._replace(). They also have two extra attributes, ._field_defaults and ._fields.

As you can see in the highlighted lines, all these additional attributes and methods have a leading underscore in their names. Why? The leading underscore prevents name conflicts with custom fields. In this case, there’s a strong reason for breaking the established convention, which suits the Python principle that practicality beats purity.

To illustrate, say that you have the following named tuple:

>>> from collections import namedtuple

>>> Car = namedtuple("Car", ["make", "model", "color", "year"])

>>> mustang = Car(make="Ford", model="Mustang", color="Red", year=2022)
>>> mustang.make
'Ford'

In this example, if you use the string "make" as a field name and the named tuple had a .make() method instead of the ._make() variation, then you’d have overridden the method with your custom field.

Note that the namedtuple() function will raise an exception if you try to name any of your fields with a leading underscore:

>>> Car = namedtuple("Car", ["make", "_model", "color", "year"])
Traceback (most recent call last):
    ...
ValueError: Field names cannot start with an underscore: '_model'

This behavior of named tuples ensures that you don’t override any of the methods available as part of a named tuple’s API.

Conclusion

Now you know about some important Python naming conventions that rely on using the underscore character (_). You’ve learned how to differentiate between public and non-public names in Python code, write safe classes for subclassing purposes, and avoid name clashes in your classes.

In this tutorial, you’ve:

• Learned a few Python naming conventions that rely on using underscores (_)
• Differentiated public vs non-public names using a single leading underscore
• Used double leading underscores to trigger name mangling in Python classes
• Explored other common uses cases of underscores in Python names

This knowledge will allow you to write code that looks Pythonic and consistent in the eyes of other Python developers. These skills are especially important when you’re writing code that you intend for other developers to work with.