Python's Built-in Functions: A Complete Exploration

Python's Built-in Functions: A Complete Exploration

by Leodanis Pozo Ramos Jul 01, 2024 basics python

Python has many built-in functions that you can use directly without importing anything. These functions cover a wide variety of common programming tasks that include performing math operations, working with built-in data types, processing iterables of data, handling input and output in your programs, working with scopes, and more.

In this tutorial, you’ll:

  • Get to know Python’s built-in functions
  • Learn about common use cases of Python’s built-in functions
  • Use these functions to solve practical problems

To get the most out of this tutorial, you’ll need to be familiar with Python programming, including topics like working with built-in data types, functions, classes, decorators, scopes, and the import system.

Take the Quiz: Test your knowledge with our interactive “Python's Built-in Functions: A Complete Exploration” quiz. You’ll receive a score upon completion to help you track your learning progress:


Interactive Quiz

Python's Built-in Functions: A Complete Exploration

Take this quiz to test your knowledge about the available built-in functions in Python. By taking this quiz, you'll deepen your understanding of how to use these functions and the common programming problems they cover, from mathematical computations to Python-specific features.

Built-in Functions in Python

Python has several functions available for you to use directly from anywhere in your code. These functions are known as built-in functions and they cover many common programming problems, from mathematical computations to Python-specific features.

In this tutorial, you’ll learn the basics of Python’s built-in functions. By the end, you’ll know what their use cases are and how they work. To kick things off, you’ll start with those built-in functions related to math computations.

In Python, you’ll find a few built-in functions that take care of common math operations, like computing the absolute value of a number, calculating powers, and more. Here’s a summary of the math-related built-in functions in Python:

Function Description
abs() Calculates the absolute value of a number
divmod() Computes the quotient and remainder of integer division
max() Finds the largest of the given arguments or items in an iterable
min() Finds the smallest of the given arguments or items in an iterable
pow() Raises a number to a power
round() Rounds a floating-point value
sum() Sums the values in an iterable

In the following sections, you’ll learn how these functions work and how to use them in your Python code.

Getting the Absolute Value of a Number: abs()

The absolute value or modulus of a real number is its non-negative value. In other words, the absolute value is the number without its sign. For example, the absolute value of -5 is 5, and the absolute value of 5 is also 5.

Python’s built-in abs() function allows you to quickly compute the absolute value or modulus of a number:

Python
>>> from decimal import Decimal
>>> from fractions import Fraction

>>> abs(-42)
42
>>> abs(42)
42

>>> abs(-42.42)
42.42
>>> abs(42.42)
42.42

>>> abs(complex("-2+3j"))
3.605551275463989
>>> abs(complex("2+3j"))
3.605551275463989

>>> abs(Fraction("-1/2"))
Fraction(1, 2)
>>> abs(Fraction("1/2"))
Fraction(1, 2)

>>> abs(Decimal("-0.5"))
Decimal('0.5')
>>> abs(Decimal("0.5"))
Decimal('0.5')

In these examples, you compute the absolute value of different numeric types using the abs() function. First, you use integer numbers, then floating-point and complex numbers, and finally, fractional and decimal numbers. In all cases, when you call the function with a negative value, the final result removes the sign.

For a practical example, say that you need to compute the total profits and losses of your company from a month’s transactions:

Python
>>> transactions = [-200, 300, -100, 500]

>>> incomes = sum(income for income in transactions if income > 0)
>>> expenses = abs(
...     sum(expense for expense in transactions if expense < 0)
... )

>>> print(f"Total incomes: ${incomes}")
Total incomes: $800
>>> print(f"Total expenses: ${expenses}")
Total expenses: $300
>>> print(f"Total profit: ${incomes - expenses}")
Total profit: $500

In this example, to compute the expenses, you use the abs() function to get the absolute value of the expenses, which results in a positive value.

Finding the Quotient and Remainder in Division: divmod()

Python provides a built-in function called divmod() that takes two numbers as arguments and returns a tuple with the quotient and remainder that result from the integer division of the input numbers:

Python
>>> divmod(8, 4)
(2, 0)

>>> divmod(6.5, 3.5)
(1.0, 3.0)

With integers as arguments, the result is the same as (a // b, a % b). With floating-point numbers, the result is (q, a % b), where q is usually math.floor(a / b), but may be 1 less than that.

As a practical example of when to use this function, say that you want to code a function that takes a time value in milliseconds and returns a string with the "00:00:00" format. Here’s a possible implementation using the divmod() function:

Python
>>> def hh_mm_ss(milliseconds):
...     seconds = round(milliseconds / 1000)
...     minutes, seconds = divmod(seconds, 60)
...     hours, minutes = divmod(minutes, 60)
...     return f"{hours:02d}:{minutes:02d}:{seconds:02d}"
...

>>> hh_mm_ss(10000)
'00:00:10'
>>> hh_mm_ss(68000)
'00:01:08'
>>> hh_mm_ss(3680000)
'01:01:20'

In this function, you first convert the input milliseconds to seconds and round the result to the nearest whole number.

Then, you use divmod() to divide the total seconds by 60 because there are 60 seconds in a minute. This computation gives you the minutes and the remaining seconds. Finally, you use divmod() again to divide the minutes by 60 because there are 60 minutes in an hour. This time, you get the hours and the remaining minutes.

Finding Minimum and Maximum Values: min() and max()

Sometimes, you need to find the smallest and largest values in an iterable or in a series of values. These can be common computations in programming, and Python has built-in functions for them.

The min() function allows you to find the minimum value in an iterable, while the max() function helps you find the maximum value. Here’s the signature of both functions:

Python
min(iterable, *[, default, key])
max(iterable, *[, default, key])

Both functions take a required argument called iterable and return the minimum and maximum values, respectively. They also take two optional keyword-only arguments:

Argument Description
default Can hold the value you want to return if the input iterable is empty
key Accepts a single-argument function to customize the comparison criteria

Here are some quick examples of how to use the min() and max() functions with different sets of arguments:

Python
>>> min([1, 2, 3, 4])
1
>>> max([1, 2, 3, 4])
4

>>> min(1, 2, 3, 4)
1
>>> max(1, 2, 3, 4)
4

>>> min([], default=0)
0
>>> max([], default=0)
0

>>> min([-2, 3, 4, -5, 1], key=abs)
1
>>> max([-2, 3, 4, -5, 1], key=abs)
-5

In the first two examples, you use min() and max() with a list of numbers. You can also use these functions with a series of positional arguments.

Then, you have two examples of using the default argument to return a suitable value when the input iterable is empty. Finally, you have two examples of using the key argument. In these examples, you use the abs() function to provide the comparison criteria.

Computing Powers: pow()

When you need to compute powers in your code, you can use the built-in pow() function. This function takes a number and raises it to a given power. Here’s the function’s signature:

Python
pow(base, exp[, mod=None])

When you call pow(), you get base to the power of exp. With these two arguments, pow() is equivalent to something like base**exp:

Python
>>> pow(2, 8)
256

>>> 2**8
256

This operation computes 2 to the power of 8, which is 256. This is equivalent to a power operation with the ** operator, which you’ll find more often in real-world code.

The mod argument allows you to do something like pow(base, exp) % mod but computed more efficiently:

Python
>>> import timeit

>>> base = 2
>>> exp = 1000000
>>> mod = 1000000

>>> timeit.timeit(
...     "pow(base, exp, mod)", globals=globals(), number=10
... ) * 1000
0.021042011212557554

>>> timeit.timeit(
...     "pow(base, exp) % mod", globals=globals(), number=10
... ) * 1000
61.956208024639636

In this example, you use the timeit() function from the timeit module to measure the computation speed. Then, you define a few variables to do the computation. In the first call to timeit(), you use the mod argument. In the second call, you use the modulo operator (%).

When you compare the resulting time consumption, you can conclude that using the mod argument is way faster than computing the power and then applying the modulo operator like in pow(base, exp) % mod.

Rounding Numbers: round()

Python’s built-in round() function takes a numeric argument and returns it rounded to a given number of digits.

The signature of round() is shown in the code below:

Python
round(number[, ndigits])

In this signature, number is typically a floating-point number, while ndigits is an optional argument that should be an integer number. This latter argument will define the precision or number of digits after the decimal point.

Here are a few examples:

Python
>>> from math import pi
>>> pi
3.141592653589793

>>> round(pi, 2)
3.14

>>> round(pi, 4)
3.1416

>>> round(pi, 6)
3.141593

In these examples, you use the pi constant from the math module and the round() function to express the number using different precision.

When you use round() with a single argument, you may get surprising results:

Python
>>> round(1.5)
2

>>> round(2.5)
2

In these two examples, the round() function rounds 1.5 up to 2 and 2.5 down to 2. This is because round() rounds to the closest multiple of 10 to the power minus ndigits. If two multiples are equally close, rounding is done toward the even choice. This rounding half to even strategy helps mitigate rounding bias. That’s why 2.5 rounds to 2 rather than 3.

Calculating Totals: sum()

Python’s built-in sum() function provides an efficient and Pythonic way to sum a list of numeric values, which is also a common intermediate step in many computations. So sum() is a pretty handy tool for a Python programmer.

The sum() function allows you to add together a series of values. Its signature is like the following:

Python
sum(iterable[, start=0])

You can call sum() with the following two arguments:

Argument Description
iterable A required argument that can hold any Python iterable.
start An optional argument that can hold an initial value.

When you call sum(), the function internally adds start plus the values in iterable. The items in the input iterable are usually numeric values. However, you can also use lists or tuples. The start argument can accept a number, list, or tuple, depending on what your iterable contains.

Here are a few examples of how to use sum() with different inputs:

Python
>>> sum([])
0

>>> sum([1, 2, 3, 4, 5])
15

>>> sum([1, 2, 3, 4, 5], 100)  # As a positional argument
115

>>> sum([1, 2, 3, 4, 5], start=100)  # As a keyword argument
115

>>> num_lists = [[1, 2, 3], [4, 5, 6]]
>>> sum(num_lists, start=[])
[1, 2, 3, 4, 5, 6]

>>> num_tuples = ((1, 2, 3), (4, 5, 6))
>>> sum(num_tuples, start=())
(1, 2, 3, 4, 5, 6)

When you call sum() with an empty iterable, you get 0 as a result because that’s the default value of start. Calling the function with a list of values returns the total sum of the provided values.

If you want to use a start value other than 0, then you can provide it as a positional or keyword argument. However, the latter approach is more readable.

The final two examples show that you can also use sum() to concatenate lists and tuples. Note that for this trick to work, you need to set start to the appropriate object. If you want to concatenate lists, then start must hold a list, and so on. Even though this trick works, the practice isn’t efficient or common. Instead, you should use the plus operator (+) for regular concatenations.

A classic example of using sum() is when you need to compute the mean or average of several numeric values. In this situation, you need to sum the input data as an intermediate step. Here’s an example:

Python mean.py
def mean(values):
    try:
        return sum(values) / len(values)
    except ZeroDivisionError:
        raise ValueError("mean() arg shouldn't be empty") from None

In this mean() function, you use sum() to sum the input values and then divide the result by the number of values in the input data.

Creating and Manipulating Basic Data Types

Python has several built-in functions that allow you to manipulate basic data types, such as integer and floating-point numbers, strings, and Boolean values. Here’s a summary of the built-in functions that help you process basic data types:

Function Description
int() Constructs an integer object from a number or string
bin() Converts an integer to a binary string
oct() Converts an integer to an octal string
hex() Converts an integer to a hexadecimal string
float() Constructs a floating-point object from a number or string
complex() Constructs a complex number from arguments
str() Creates a string object
repr() Creates a developer-friendly string representation of an object
bool() Converts an argument to a Boolean value
ord() Looks up the integer code point of a character
chr() Looks up the character for the given integer code point
bytes() Creates a bytes object (similar to bytearray, but immutable)
bytearray() Creates an object of the bytearray class

In the following sections, you’ll learn the basics of working with these functions and how to use them in your Python code.

Representing Integer Numbers: int(), bin(), oct(), and hex()

Integer numbers are pretty useful in programming. Python has a built-in data type called int that represents integers. When working with integers, sometimes you need to express them in different bases like 2, 8, or 16. You may also need to convert strings or other numeric types to integers.

For the latter task, you can use the built-in int() function. Here are some examples of using it:

Python
>>> int()
0

>>> int(42.42)
42

>>> int("42")
42

>>> int("42.42")
Traceback (most recent call last):
    ...
ValueError: invalid literal for int() with base 10: '42.42'

>>> int("one")
Traceback (most recent call last):
    ...
ValueError: invalid literal for int() with base 10: 'one'

With no argument, int() returns 0. This behavior is especially useful when you need a factory function for classes like defaultdict from the collections module. With floating-point numbers, int() just removes the decimal part and returns the whole part. Finally, with a string as an argument, int() returns the corresponding integer only if the string represents a valid integer number.

You can also use int() to convert a binary, octal, or hexadecimal string representation into an integer number:

Python
>>> int("0b10", base=2)
2

>>> int("0o10", base=8)
8

>>> int("0x10", base=16)
16

In the first example, you use int() to convert a string representing a number in binary format to its equivalent decimal integer. Note that for this operation to work, you need to set the base argument to the appropriate base, which is 2 for binary numbers. Next, you do similar conversions with octal and hexadecimal strings. Again, you have to set base to the appropriate value.

The bin(), oct(), and hex() functions allow you to do the opposite operation. With them, you can convert a given integer into its binary, octal, or hexadecimal representation:

Python
>>> bin(42)
'0b101010'

>>> oct(42)
'0o52'

>>> hex(42)
'0x2a'

In these examples, you use an integer number as an argument to bin(), oct(), and hex(). As a result, you get the string representation of the input value in binary, octal, and hexadecimal format, respectively.

Manipulating Other Numbers: float() and complex()

Python has basic built-in types to represent floating-point and complex numbers. These types have associated built-in functions for conversion purposes. So, for floating-point numbers, you have the float() function, and for complex numbers you have complex().

Here are the signatures of both functions:

Python
float(number=0.0)

complex(real=0, imag=0)
complex(string)

The float() function takes a single argument representing a numeric value. This argument accepts numbers or strings that represent valid numbers:

Python
>>> float()
0.0

>>> float(42)
42.0

>>> float("42")
42.0

>>> float("3.14")
3.14

>>> float("one")
Traceback (most recent call last):
    ...
ValueError: could not convert string to float: 'one'

With no arguments, float() returns 0.0. With integer numbers, it returns the equivalent floating-point number with 0 as the decimal part. With strings representing numbers, float() returns the equivalent floating-point number. However, it fails if the input string doesn’t represent a valid numeric value.

The complex() function allows you to work with complex numbers. This function has two different signatures. The first signature has two arguments:

Argument Description
real The number’s real part
imag The number’s imaginary part

These arguments accept numeric values, such as integer or floating-point numbers. Here’s how this variation of complex() works:

Python
>>> complex(3, 6)
(3+6j)

>>> complex(1, 0)
(1+0j)

>>> complex(0, 1)
1j

>>> complex(3.14, -2.75)
(3.14-2.75j)

You can call complex() with numeric values, resulting in a complex number. Note that Python uses a j to define the imaginary part.

The second signature of complex() takes a single argument that should be a string:

Python
>>> complex("3+6j")
(3+6j)

>>> complex("1+0j")
(1+0j)

>>> complex("1j")
1j

>>> complex("3.14-2.75j")
(3.14-2.75j)

When you use strings to create complex numbers with complex(), you have to make sure that the input string has a valid format. It should consist of the real part, the sign, and the imaginary part. You can’t add spaces to separate these components.

Building and Representing Strings: str() and repr()

When it comes to creating and working with Python strings, you have two fundamental built-in functions to consider:

  1. str()
  2. repr()

With the str() function, you can create new strings or convert existing objects to strings:

Python
>>> str()
''

>>> str(42)
'42'

>>> str(3.14)
'3.14'

>>> str([1, 2, 3])
'[1, 2, 3]'

>>> str({"one": 1, "two": 2, "three": 3})
"{'one': 1, 'two': 2, 'three': 3}"

>>> str({"A", "B", "C"})
"{'B', 'C', 'A'}"

In the first example, you use str() without an argument to create an empty string. In the other examples, you get strings with user-friendly representations of the input objects.

For a practical use case, say that you have a list of numeric values and want to join them using the str.join() method, which only accepts iterables of strings. In this case, you can do something like the following:

Python
>>> "-".join(str(value) for value in [1, 2, 3, 4, 5])
'1-2-3-4-5'

In this example, you use a generator expression to convert each number to its string representation before calling .join(). This way, you avoid getting an error in your code.

For its part, the built-in repr() function gives you a developer-friendly representation of the object at hand:

Python
>>> repr(42)
'42'

>>> repr(3.14)
'3.14'

>>> repr([1, 2, 3])
'[1, 2, 3]'

>>> repr({"one": 1, "two": 2, "three": 3})
"{'one': 1, 'two': 2, 'three': 3}"

>>> repr({"A", "B", "C"})
"{'B', 'C', 'A'}"

For built-in types, the string representation you get with repr() is the same as the one you get with the str() function.

To see the difference between str() and repr(), consider the following example that uses the datetime module:

Python
>>> import datetime
>>> today = datetime.datetime.now()

>>> repr(today)
'datetime.datetime(2024, 7, 1, 12, 38, 53, 180208)'

>>> str(today)
'2024-07-01 12:38:53.180208'

The repr() method gives you a developer-friendly string representation of the datetime object. Ideally, you should be able to re-create the object using this representation. In other words, you should be able to copy and paste the resulting representation to re-create the object. That’s why this string representation is said to be developer-friendly.

In contrast, the string representation that you get from calling str() should aim to be readable and informative for end users.

Processing Boolean Values: bool()

Python’s built-in bool() function allows you to determine the truth value of any Python object. It’s a predicate function because it always returns True or False. To figure out if an object is falsy, in other words, whether bool() returns False when applied to the object, Python uses the following internal rules:

  • Constants that are defined to be false: None and False
  • The zero of any numeric type: 0, 0.0, 0j, Decimal(0), Fraction(0, 1)
  • Empty sequences and collections: '', (), [], {}, set(), range(0)

The rest of the objects are considered truthy in Python. Custom objects are considered truthy by default unless they provide a .__bool__() special method that defines a different behavior.

Here are a few examples of how bool() works:

Python
>>> bool()
False

>>> bool(0)
False
>>> bool(42)
True

>>> bool(0.0)
False
>>> bool(3.14)
True

>>> bool("")
False
>>> bool("Hello")
True

>>> bool([])
False
>>> bool([1, 2, 3])
True

In the first example, you call bool() without an argument and get False as a result. In the rest of the examples, you can confirm that Python consistently applies the rules listed above. In practice, you’ll only need to use bool() when your code explicitly requires a Boolean value instead of a different object.

As an example of using bool(), say that you have the following implementation of a stack data structure:

Python stack.py
class Stack:
    def __init__(self, items=None):
        self.items = list(items) if items is not None else []

    def push(self, item):
        self.items.append(item)

    def pop(self):
        return self.items.pop()

    def __bool__(self):
        return bool(self.items)

In this example, your Stack class implements the .__bool__() special method to support Boolean operations on its objects. This method guarantees that when a given Stack object is empty, the bool() function returns False and True otherwise. Here’s an example:

Python
>>> from stack import Stack

>>> stack = Stack()

>>> bool(stack)
False

>>> stack.push(4)

>>> bool(stack)
True

In this code snippet, you first create an empty stack. When you pass this object to bool(), you get False. Then, you push a value into the stack and call bool() again. This time, you get True because the stack isn’t empty anymore.

Encoding Strings: ord() and chr()

Character encoding is an important topic in most programming languages. In Python, strings use the Unicode characters set by default. Each Unicode character has an associated code point, which is an integer number. To get the code point of a given character, you can use the built-in ord() function:

Python
>>> ord("A")
65

>>> ord("Z")
90

>>> ord("x")
120

>>> ord("ñ")
241

>>> ord("&")
38

Every Unicode character has an associated code point that uniquely identifies the character in the Unicode table. In these examples, you use the function to get the code point of a few characters.

In practice, you can use the ord() function to implement basic cryptographic techniques, sort strings or characters, validate input characters, and so on. Here’s a quick toy example of a function that only checks whether all the characters in a string are uppercase letters of the English alphabet:

Python
>>> def is_uppercase(text):
...     for char in text:
...         if not (65 <= ord(char) <= 90):
...             return False
...     return True
...

>>> is_uppercase("HELLO")
True

>>> is_uppercase("Hello")
False

In this function, you use ord() to determine whether the characters in a string are between 65 and 90, which is the interval of code points for uppercase letters, A to Z, in the Unicode table.

Sometimes, you may need to determine the code point that identifies a given Unicode character. In this situation, you can use the built-in chr() function:

Python
>>> chr(65)
'A'

>>> chr(90)
'Z'

>>> chr(120)
'x'

>>> chr(241)
'ñ'

>>> chr(38)
'&'

The chr() function does the opposite operation of ord(). It allows you to find the code point associated with a specific character.

The ord() and chr() functions are sort of complementary, and therefore, you’ll probably find them used together in tandem.

Creating Bytes and Byte Arrays: bytes() and bytearray()

Python’s bytes and byte arrays are built-in types that Python provides out of the box to manipulate binary data, encode and decode text, process file input and output, and communicate through networks.

The bytes data type is immutable, while the bytearray type is mutable. To create objects derived from these data types, you can use the built-in bytes() and bytearray() functions.

The bytes() and bytearray() functions have the following signatures:

Python
bytes(source=b"")
bytes(source, encoding)
bytes(source, encoding, errors)

bytearray(source=b"")
bytearray(source, encoding)
bytearray(source, encoding, errors)

Both functions have three different signatures. The first signature of both functions accepts a bytes literal as an argument. These literals are similar to string literals, but they start with a b and only accept ASCII characters.

Here’s a summary of the arguments and their meaning:

Argument Description
source A bytes literal or a string
encoding The character encoding to use for decoding source if it holds a string
errors A handler for encoding and decoding errors

The encoding argument is only required if the source argument is a string, in which case, you must provide the appropriate encoding so that Python can convert the string into bytes.

Finally, the errors arguments is also optional and should hold one of the following error handlers:

Handler Description
"strict" Raises a UnicodeDecodeError or UnicodeEncodeError exception when encoding problems appear
"ignore" Ignores the characters that can’t be encoded
"replace" Replaces the characters that can’t be encoded with a question mark (?)
"xmlcharrefreplace" Replaces the characters that can’t be encoded with an XML character reference
"backslashreplace" Replaces the characters that can’t be encoded with Python’s string backslash escape sequences

By choosing the appropriate error handlers, you can set up a good strategy for those situations when you call the bytes() and bytearray() functions with erroneous data.

Here are a few examples of using the bytes() and bytearray() functions:

Python
>>> bytes()
b''

>>> bytes(b"Using ASCII characters or bytes \xc3\xb1")
b'Using ASCII characters or bytes \xc3\xb1'

>>> bytes("Using non-ASCII characters: ñ Ł", encoding="utf-8")
b'Using non-ASCII characters: \xc3\xb1 \xc5\x81'

>>> bytearray()
bytearray(b'')

>>> bytearray(b"Using ASCII characters or bytes \xc3\xb1")
bytearray(b'Using ASCII characters or bytes \xc3\xb1')

>>> bytearray("Using non-ASCII characters: ñ Ł", encoding="utf-8")
bytearray(b'Using non-ASCII characters: \xc3\xb1 \xc5\x81')

In these examples, you create bytes and bytearray objects using bytes literals, and strings with the correct encoding as an argument. Note that you can call the bytes() function with no arguments to create an empty bytes object.

Now consider the following examples that show how to use error handlers:

Python
>>> bytes(
...     "Using non-ASCII characters with the ASCII encoding: ñ Ł",
...     encoding="ascii"
... )
Traceback (most recent call last):
    ...
UnicodeEncodeError: 'ascii' codec can't encode character '\xf1'
    in position 52: ordinal not in range(128)

>>> bytes(
...     "Using non-ASCII characters with the ASCII encoding: ñ Ł",
...     encoding="ascii",
...     errors="ignore"
... )
b'Using non-ASCII characters with the ASCII encoding:  '

>>> bytes(
...     "Using non-ASCII characters with the ASCII encoding: ñ Ł",
...     encoding="ascii",
...     errors="replace"
... )
b'Using non-ASCII characters with the ASCII encoding: ? ?'

>>> bytes(
...     "Using non-ASCII characters with the ASCII encoding: ñ Ł",
...     encoding="ascii",
...     errors="xmlcharrefreplace"
... )
b'Using non-ASCII characters with the ASCII encoding: &#241; &#321;'

>>> bytes(
...     "Using non-ASCII characters with the ASCII encoding: ñ Ł",
...     encoding="ascii",
...     errors="backslashreplace"
... )
b'Using non-ASCII characters with the ASCII encoding: \\xf1 \\u0141'

In these examples, you only use bytes() because bytearray() would work similarly. The only difference is that bytes() returns immutable objects while bytearray() returns mutable ones.

These examples use non-ASCII characters with the ASCII encoding, which will cause encoding errors that you’ll need to handle. The default value of the errors argument is "strict". That’s why you get a UnicodeEncodeError exception in the first example above.

Then you set errors to "ignore" so that Python ignores any encoding error. In this case, the ñ and Ł characters are removed. If you set errors to "replace", then ñ and Ł are each replaced with a question mark.

Using "xmlcharrefreplace" as the error handler makes Python replace the ñ and Ł characters with their respective XML character reference. Finally, using "backslashreplace" escapes the problematic characters by using the appropriate escape sequence.

Creating Collection Data Types

A fundamental feature of Python is the rich set of collection data types built into the language. You’ll have several built-in functions that allow you to manipulate these data types, which include lists, tuples, dictionaries, sets, and bytes.

Here’s a summary of the built-in functions that help you process collection data types:

Function Description
list() Creates a list object from an iterable
tuple() Creates a tuple object from an iterable
dict() Creates a dict object from a series of key-value pairs or keyword arguments
set() Creates a set object from an iterable
frozenset() Creates a frozenset object from an iterable

In the following sections, you’ll learn the basics of working with these functions and using them to create and manipulate collections in your Python code.

Creating Lists and Tuples: list() and tuple()

Python’s list is a fundamental built-in data type with an impressive set of features. Lists are mutable and allow you to efficiently organize and manipulate data that can be heterogeneous but is typically homogeneous. For example, you can use a list to store a column from a database table.

Similarly, Python’s tuple is another built-in type. Unlike lists, tuples are immutable. You can use them to organize data that can be homogeneous but is typically heterogeneous. For example, you can use a tuple to store a row from a database table.

Python’s built-in list() and tuple() functions allow you to create list and tuple objects.

The list() function takes an iterable as an argument and returns a list object built out of the input data. So, its signature looks something like the following:

Python
list([iterable])

Note that the square brackets around iterable mean that the argument is optional, so the brackets aren’t part of the syntax.

Here are a few examples of using list() to create list objects:

Python
>>> list()
[]

>>> list("Hello")
['H', 'e', 'l', 'l', 'o']

>>> list((1, 2, 3, 4, 5))
[1, 2, 3, 4, 5]

>>> list({"circle", "square", "triangle", "rectangle", "pentagon"})
['square', 'rectangle', 'triangle', 'pentagon', 'circle']

>>> list({"name": "John", "age": 30, "city": "New York"})
['name', 'age', 'city']
>>> list({"name": "John", "age": 30, "city": "New York"}.keys())
['name', 'age', 'city']

>>> list({"name": "John", "age": 30, "city": "New York"}.values())
['John', 30, 'New York']

>>> list({"name": "John", "age": 30, "city": "New York"}.items())
[('name', 'John'), ('age', 30), ('city', 'New York')]

When you call list() without an argument, you create a new empty list. When you use a string as an argument, you create a list of characters. When you use a tuple, you convert the tuple into a list.

The list() function even accepts sets, but you need to remember that sets are unordered data structures, so you won’t be able to predict the final order of items in the resulting list.

When it comes to using dictionaries with list(), you have four possibilities. You can create a list of keys using the dictionary directly or using its .keys() method. If you want to create a list of values, then you can use the .values() method. Finally, if you want to create a list of key-value pairs, then you can use the .items() method.

Lists have many use cases in Python code. They’re flexible, powerful, and feature-full, so you’ll find them in almost every piece of Python code.

Tuples are commonly used to store heterogeneous and immutable data. The tuple() function allows you to create tuples on the fly. Here’s the signature:

Python
tuple([iterable])

The square brackets around iterable mean that the argument is optional, so the brackets aren’t part of the syntax.

Consider the following examples of using tuple() in your code:

Python
>>> tuple()
()

>>> tuple("Hello")
('H', 'e', 'l', 'l', 'o')

>>> tuple(["Jane Doe", 25, 1.75, "Canada"])
('Jane Doe', 25, 1.75, 'Canada')

>>> tuple({
...     "manufacturer": "Ford",
...     "model": "Mustang",
...     "color": "Blue",
... }.values())
('Ford', 'Mustang', 'Blue')

You can use tuple() with no arguments to create an empty tuple. This will be more readable than using an empty pair of parentheses (). When you pass a string into tuple(), you get a tuple of characters.

In the third example, you use tuple() to convert a list of heterogeneous data into a tuple, which would be a more appropriate data structure for storing this type of data. Finally, you use the values of a dictionary to build a tuple.

Just like lists, tuples are pretty useful in Python. You’ll see them used in many use cases, especially in those situations where you need to store immutable heterogeneous data.

Constructing Dictionaries: dict()

Dictionaries are a fundamental built-in data structure in Python. They’re everywhere and a core part of the language itself. You’ll find many use cases for dictionaries in your code. As for other built-in collections, Python also has a built-in function that allows you to create dictionaries: the dict() function.

The dict() function has the following signatures:

Python
dict(**kwargs)
dict(mapping, **kwargs)
dict(iterable, **kwargs)

All these signatures accept what is known as keyword arguments (**kwargs) or named arguments. The second signature takes a mapping, which can be another dictionary. Finally, the third signature accepts an iterable of key-value pairs, which can be a list of two-item tuples, for example.

Here are some quick examples of using the dict() function in different ways:

Python
>>> dict()
{}

>>> jane = dict(name="Jane", age="30", country="Canada")
>>> jane
{'name': 'Jane', 'age': '30', 'country': 'Canada'}

>>> dict(jane, job="Python Dev")
{'name': 'Jane', 'age': '30', 'country': 'Canada', 'job': 'Python Dev'}

>>> dict([("name", "Jane"), ("age", 30), ("country", "Canada")])
{'name': 'Jane', 'age': 30, 'country': 'Canada'}

Again, when creating an empty dictionary, you can use the dict() function without arguments. This is less common than using a pair of curly brackets {}, but again, it can be more readable and explicit in some contexts.

Then, you create a jane dictionary using keyword arguments. This is a clean and elegant way to build dictionaries in Python.

The third example shows how you can combine a mapping with keyword arguments to create a new dictionary object. Finally, in the fourth example, you create a new dictionary from a list of tuples.

Creating Sets and Frozen Sets: set() and frozenset()

Python’s set is a built-in data type for creating collections of unique and hashable objects, typically called elements or members. In Python, sets support the operations defined for mathematical sets, including union, difference, symmetric difference, and others.

Python has two types of sets:

  1. set
  2. frozenset

The difference between these two data types is that set objects are mutable, and frozenset objects are immutable.

As with other data types, Python also provides built-in functions for creating sets and frozen sets. You’ll have the set() and frozenset() functions, respectively. The signature for these functions is shown below:

Python
set([iterable])
frozenset([iterable])

Again, the square brackets indicate that the input iterable is optional. Now check out the following examples of creating sets and frozen sets:

Python
>>> set()
set()

>>> frozenset()
frozenset()

>>> set(["square", "rectangle", "triangle", "pentagon", "circle"])
{'square', 'triangle', 'circle', 'rectangle', 'pentagon'}

>>> frozenset(["square", "rectangle", "triangle", "pentagon", "circle"])
frozenset({'square', 'triangle', 'circle', 'rectangle', 'pentagon'})

>>> set(("red", "green", "blue", "red"))
{'green', 'red', 'blue'}

>>> frozenset(("red", "green", "blue", "red"))
frozenset({'green', 'red', 'blue'})

When you call set() and frozenset() without arguments, you create an empty set or frozen set, respectively. In the case of sets, you don’t have a literal that you can use to create an empty set because a pair of curly brackets ({}) defines an empty dictionary. So, to create an empty set, you must use the set() function.

In the rest of the examples, you use iterables, such as lists and tuples, to create sets and frozen sets. It’s important to note that when the input iterable has repeated elements, the final set will have a single instance of the repeated item. Also, sets are unordered data structures, so you won’t be able to predict the final order of items in the resulting set when providing a list.

Processing Iterables and Iterators

Python’s iterators and iterables are two different but related tools that come in handy when you need to iterate over a data stream or collection of data. Iterators power and control the iteration process, while iterables typically hold data that can be iterated over one value at a time.

Python has several built-in functions that you can use to work with iterables and iterators. Here’s a summary of these functions:

Function Description
len() Calculates the length of a sized object
reversed() Constructs a reverse iterator
sorted() Creates a sorted list from an iterable
all() Checks if all elements of an iterable are true
any() Checks if any elements of an iterable are true
range() Generates a range of integer values
enumerate() Creates an iterator of tuples containing indices and values from an iterable
slice() Creates a slice object
zip() Creates an iterator that aggregates elements from iterables
iter() Constructs an iterator object
next() Retrieves the next item from an iterator
filter() Filters elements from an iterable
map() Applies a function to every item of an iterable

In the following sections, you’ll learn about all these built-in functions and how they can be useful when processing iterables and iterators in your Python code.

Determining the Number of Items in a Container: len()

One of the most common operations that you’ll perform on collections is to determine the number of items stored in an existing sequence or collection. To complete this task, Python has the built-in len() function.

The len() function takes a single argument, which can be a sequence, such as a string, tuple, or list. It can also be a collection, such as a dictionary, set, or frozen set. The function returns the length or number of items of the input object.

Here are a few examples of using len() with different objects:

Python
>>> len("Python")
6

>>> len(("Jane Doe", 25, 1.75, "Canada"))
4

>>> len([1, 2, 3, 4, 5])
5

>>> len({"green", "red", "blue"})
3

>>> len({"name": "Jane", "age": 30, "country": "Canada"})
3

In the first example, you use len() to get the number of characters in a string. Then, you use the function to determine the length of a tuple, list, and set. Finally, you use len() with a dictionary as an argument. In this case, you get the number of key-value pairs in the input dictionary.

Note that len() returns 0 when you call it with empty containers:

Python
>>> len("")
0
>>> len(())
0
>>> len([])
0

In these examples, you use len() with an empty string, tuple, and list. In all cases, you get 0 as a result.

The len() function can be useful in several situations. A common example is when you need to compute the average of a series of numeric values:

Python
>>> grades = [90, 97, 100, 87]

>>> sum(grades) / len(grades)
93.5

In this example, len() gives you the number of values in the list of grades. Then, you use this value to compute the average grade.

Reversing and Sorting Iterables: reversed() and sorted()

Reversing and sorting the values in an iterable is another useful operation in programming. Because these operations are so common, Python has built-in functions for them. When you want to reverse an iterable, then you can use the built-in reversed() function.

The reversed() function takes an iterable as an argument and returns an iterator that yields the items in reverse order:

Python
>>> reversed([0, 1, 2, 3, 4, 5])
<list_reverseiterator object at 0x107062b30>

>>> list(reversed([0, 1, 2, 3, 4, 5]))
[5, 4, 3, 2, 1, 0]

In this example, you call reversed() with a list of numbers. The function returns a reverse iterator. To display its content, you can use the list() function to build a list from the iterator.

Similarly, when you need to sort the values in an iterable, you can use the sorted() function. This function has the following signature:

Python
sorted(iterable, key=None, reverse=False)

The first argument is an iterable object, such as a string, list, tuple, or dictionary. Then, you have the key and reverse arguments, which have the following meanings:

Argument Description
key Specifies a one-argument function that extracts a comparison key from each element in the input iterable
reverse Is a Boolean value that allows you to sort the items in reverse order if you set it to True

It’s important to note that the key argument accepts function objects. In other words, functions without the calling parentheses.

Here are a few examples of using sorted() with different built-in containers as arguments:

Python
>>> sorted("bdeac")
['a', 'b', 'c', 'd', 'e']

>>> sorted([4, 2, 7, 5, 1, 6, 3])
[1, 2, 3, 4, 5, 6, 7]

>>> sorted([4, 2, 7, 5, 1, 6, 3], reverse=True)
[7, 6, 5, 4, 3, 2, 1]

Unlike reversed(), the sorted() function always returns a list object rather than an iterator. In the first example, you use a string as an argument to sorted() and get a list of characters in alphabetical order. Under the hood, Python uses the character’s Unicode code point to compare strings.

In the third example, you set the reverse argument to True and get the list of numbers sorted in reverse order.

Finally, the key argument may be useful in several situations. A common use case for this argument is when you need to sort items that are also containers. Here’s an example of sorting a list of two-value tuples:

Python
>>> points = [(1, 2), (3, 1), (4, 0), (2, 1)]

>>> sorted(points, key=lambda point: point[0])
[(1, 2), (2, 1), (3, 1), (4, 0)]

>>> sorted(points, key=lambda point: point[1])
[(4, 0), (3, 1), (2, 1), (1, 2)]

In the first highlighted line, you use a lambda function that takes a point as an argument and returns its first coordinate. This example would produce the same result if you call sorted() without key because of the way Python compares tuples. In the second highlighted line, the lambda function returns the second coordinate. These functions provide comparison keys for the sorting processes.

So, in the first example, you sort the points by their first coordinate, while in the second example, you sort points by their second coordinate.

Determining the Truth Value of Items in Iterables: all() and any()

Sometimes, you may need to determine whether all the items in an iterable are true. To do this, Python has the built-in all() function. At other times, you may need to find out if at least one item in an iterable is true. For this purpose, Python has the built-in any() function.

The signature of all() and any() are shown below:

Python
all(iterable)

any(iterable)

Both functions take an iterable of objects as an argument. The all() function returns True when all the items in the input iterable are true and False when at least one item is false.

Here are a few examples of how all() works:

Python
>>> all([1, 2, 3, 4])
True
>>> all([1, 2, 3, 4, 0])
False

>>> all(["Hello", ""])
False
>>> all(["Hello", "World"])
True

In the first two examples, you use all() with a list of numbers. The first list contains integer values different from 0, which Python considers true. So, all() returns True. Then, in the second example, you add a 0 to the list, and all() returns False because this value is considered false in Python.

In the final two examples, you use a list of strings. Python considers an empty string false, so all() returns False. Finally, you pass a list with two non-empty strings, and all() reruns True.

You may find several use cases for all() in real-world code. Perhaps passing a generator expression as an argument to all() is one of the most powerful ways to use the function. For example, say that you want to determine whether all the numbers in a list are in a given interval. In this situation, you can use all() like in the following code:

Python
>>> numbers = [10, 5, 6, 4, 7, 8, 18]

>>> # Between 0 and 10
>>> all(0 <= x <= 10 for x in numbers)
False

>>> # Between 0 and 20
>>> all(0 <= x <= 20 for x in numbers)
True

In the first highlighted line, you use a generator expression as an argument to all(). The generator checks whether the numbers are between 0 and 10 and generates Boolean values according to the condition’s result.

The all() function checks if all the generated values are True. If that’s the case, then you get True. On the other hand, if at least one of the generated values is False, then you get False.

In contrast to all(), the any() function returns True if at least one item is true and False if all the items are false. So, you can use any() when you need to determine if at least one item in an iterable is true.

Here are a couple of examples of using any() in Python:

Python
>>> any([0, 0.0, False, "", []])
False

>>> any([0, 0.0, False, "", [], 42])
True

In the first example, all the objects in the input list are false according to Python’s internal rules for determining the truth value of an object. Because all the objects are false, any() returns False. In the second example, you add the number 42 to the end of the input list. Because 42 is truthy, any() returns True.

Again, just like all(), the any() function can shine when you use it with a generator expression that checks for some condition. For example, say that you want to know if at least one letter in a given text is in uppercase. In this situation, you can do something like the following:

Python
>>> any(letter.isupper() for letter in  "hello, world!")
False

>>> any(letter.isupper() for letter in  "Hello, World!")
True

In these examples, you use the str.isupper() method to determine whether a letter is in uppercase. This method returns True or False, so you get an iterable of Boolean values. The job of any() is to determine if any of these values is true.

In the first example, you have no uppercase letters, so any() returns False. In the second example, you have an uppercase H, so any() returns True.

Creating Ranges of Integer Values: range()

Sometimes, you need to build numerical ranges to represent a series of integer values in a given interval. Usually, you need the numbers to be consecutive, but you may also want them to be nonsequential.

For example, you can create a range of values that contains all the digits using a Python list:

Python
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

This approach will work if your range is relatively small like the one in this example. However, what if your range needs to have a million values? Building that kind of range with a list would be a difficult task. There’s a better way.

Python has the built-in range() function to facilitate the creation of numerical ranges:

Python
>>> range(10)
range(0, 10)

>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> range(1_000_000)
range(0, 1000000)

>>> list(range(1_000_000))
[1, 2, 3,
 ...,
 999998, 999999]

The range() function returns a range object rather than a list. If you want to check the values in a concrete range, then you can wrap it in a call to list(), as you did in the second example. Now you have a list of digits.

In the third example, you create a range with a million integers from 0 to 999999. If you want to see it in action, pass it to list(). You’ll get a terminal window full of numbers!

The built-in range() function has the following signatures:

Python
range(stop)
range(start, stop, step=1)

You have a one-argument and three-argument variant of the function. Here’s what each argument means:

Argument Description
start It holds the initial value in the range. It defaults to 0 and is included in the range.
stop It holds the value at which the range stops. It’s a required argument, and its value isn’t included in the range.
step It holds the step through successive values. It’s an optional argument that defaults to 1.

So far, you’ve used the one-argument variation, which starts the range at 0 and builds a range of consecutive integers. Here are a couple of examples showing the three-argument variation:

Python
>>> list(range(1, 11))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

>>> list(range(10, 101, 10))
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100]

Note that in the three-argument variation, the third argument is optional. This means that you can call the function with two arguments and rely on the default value of step, which is 1. That’s what you do in the first example, which builds a range of consecutive integers from 1 to 11. Again, 11 isn’t included in the final range because it’s the value at which range() stops issuing values.

In the second example, you create a range that starts at 10 and goes up to 101 with a step of 10.

You can also use negative values for the arguments of range(). A common use case for this is when you need to create ranges of negative numbers and ranges that count backward:

Python
>>> list(range(-10, 0))
[-10, -9, -8, -7, -6, -5, -4, -3, -2, -1]

>>> list(range(100, 0, -10))
[100, 90, 80, 70, 60, 50, 40, 30, 20, 10]

In this example, the first range contains consecutive negative numbers from -10 up to 0. The second range counts backward from 100 to 0 with a step of -10.

Ranges can be useful in several situations. A good use case for a range is when you need to perform an operation a given number times:

Python
>>> for _ in range(3):
...     print("Beep")
...
Beep
Beep
Beep

This loop runs three times and prints a message to the screen. However, the loop doesn’t need to use the loop variable in its body. So, you use an underscore to denote that this is a throwaway variable.

A common misuse of range objects in Python is to use them as a way to loop over an existing iterable:

Python
>>> letters = ["a", "b", "c", "d"]

>>> for index in range(len(letters)):
...     print(index, letters[index])
...
0 a
1 b
2 c
3 d

This kind of for loop isn’t the best example of a Pythonic construct. It uses range() to loop over the letters with numeric indices. In the following section, you’ll learn about the Pythonic way to write this kind of loop.

Enumerating Items in Loops: enumerate()

A Pythonic for loop iterates over the items in an iterable without considering the index at which a given item lives or the order in which the items were processed. You’ll find that most Python for loops will look something like the following:

Python
>>> colors = ["red", "orange", "yellow", "green"]

>>> for color in colors:
...     print(color)
...
red
orange
yellow
green

This way of writing loops in Python is explicit and intuitive, which deeply impacts the loop’s readability. However, sometimes you’ll need a way to access the index where a given item lives in the input iterable.

Often, when you’re starting with Python and need to access indices in a loop, you might end up writing a loop like the following:

Python
>>> for index in range(len(colors)):
...     print(index, colors[index])
...
...
0 red
1 orange
2 yellow
3 green
4 blue
5 indigo
6 violet

This kind of loop works. However, it’s not what you can call a Pythonic loop. It uses a couple of tricks to somehow mimic iteration over indices. Python offers a better tool to do this, and that tool is the built-in enumerate() function.

Here’s how you’ll write the above loop using the enumerate() function:

Python
>>> for index, color in enumerate(colors):
...     print(index, color)
...
0 red
1 orange
2 yellow
3 green
4 blue
5 indigo
6 violet

This loop is readable and explicit. The enumerate() function takes an iterable as an argument and generates two-item tuples containing an integer index and the associated item.

Here’s the function’s signature:

Python
enumerate(iterable, start=0)

The first argument is an iterable object. The second argument, start, gives you the option to define a starting value for the enumeration. The default value is 0 because that’s the usual start value of indices in programming. However, in some situations, using a different starting point, like 1, can be convenient.

To illustrate how to use the start argument, say that you’re building a text-based user interface (TUI) application and want to display a menu with some options. The options should have an associated number so that the user can choose the desired action. In this situation, you can use enumerate() as in the code below:

Python
>>> def list_menu(options):
...     print("Main Menu:")
...     for index, option in enumerate(options, start=1):
...         print(f"{index}. {option}")
...

>>> list_menu(["Open", "Save", "Settings", "Quit"])
Main Menu:
1. Open
2. Save
3. Settings
4. Quit

From the end user’s perspective, starting the menu list at 1 is the natural way to go. You can achieve this effect by setting start to 1 in the call to enumerate(). Now, the menu starts at 1 instead of 0.

Extracting Slices or Portions of Sequences: slice()

When you’re working with Python, you may need to extract a portion or a slice of an existing sequence, such as a string, list, or tuple. To do this, you typically use the slicing operator ([]). However, you can also use the built-in slice() function. The slice() function has the following signatures:

Python
slice(stop)
slice(start, stop, step=None)

The slice() function returns a slice object representing the set of indices specified by range(start, stop, step). The arguments here have a similar meaning as in the range() function:

Argument Description
start It holds the initial value in the slice. It defaults to None, which indicates the start of the sequence.
stop It holds the value at which the slice stops. It’s a required argument, and its value isn’t included in the slice. When set to None, it means the end of the sequence (len(sequence)).
step It holds the step through successive values. It’s an optional argument that defaults to None, which means a step of 1.

Slices don’t represent numeric ranges but sets of indices. You can use these indices to extract a portion of a list. Here are a few examples:

Python
>>> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> even = numbers[slice(1, None, 2)]
>>> even
[2, 4, 6, 8]

>>> odd = numbers[slice(None, None, 2)]
>>> odd
[1, 3, 5, 7, 9]

In the first slice, you start at 1 and go up to the end of the list with a step of 2. This slice gives you a list of even numbers. In the second example, you start at the beginning of the list and go up to the end of the list with a step of 2. With this slice, you get a list of odd numbers.

Note that in most Python code, you won’t see or use slice() the way you did in the example above. In most cases, you’ll use the slicing operator, [start:stop:step]. Here’s what this looks like with this operator:

Python
>>> even = numbers[1::2]
>>> even
[2, 4, 6, 8]

>>> odd = numbers[::2]
>>> odd
[1, 3, 5, 7, 9]

In these examples, you use the slicing operator to get the even and odd numbers from your original list. Note that skipping a given index makes that index rely on its default value. For example, when you don’t provide a start index, then it defaults to the beginning of the list.

Zipping Iterables for Parallel Iteration: zip()

Python’s built-in zip() function allows you to iterate over multiple iterables in parallel. This function creates an iterator that will aggregate elements from two or more iterables, yielding tuples of values.

Here’s the signature of the built-in zip() function:

Python
zip(*iterables, strict=False)

This function takes an undefined number of iterables as arguments and yields tuples of items on demand. The tuples will contain an item from each input iterable, making it ideal for parallel iteration.

Here are some quick examples:

Python
>>> letters = ["a", "b", "c"]
>>> numbers = [1, 2, 3]
>>> operators = ["*", "/", "+"]

>>> for characters in zip(letters, numbers, operators):
...     print(characters)
...
('a', 1, '*')
('b', 2, '/')
('c', 3, '+')

>>> for l, n, o in zip(letters, numbers, operators):
...     print(f"{l} {n} {o}")
...
a 1 *
b 2 /
c 3 +

In the first loop, you use a single loop variable to store each tuple that you get from zip(). The first tuple contains the first items of each input iterable. The second tuple contains the second items, and so on. In the second loop, you use three loop variables to unpack the items of each generated tuple.

A nice use case of zip() is when you have two lists and want to create a dictionary from them. Consider the following example:

Python
>>> keys = ["name", "age", "country"]
>>> values = ["Jane", "30", "Canada"]

>>> dict(zip(keys, values))
{'name': 'Jane', 'age': '30', 'country': 'Canada'}

In this example, you combine two existing lists using zip() and pass the resulting tuples to the dict() function to create a dictionary.

The strict argument to zip() was added in Python 3.10 and is a keyword-only argument that provides a safe way to handle iterables of unequal length. The argument’s default value is False, which means that zip() will only generate as many tuples as items in the shortest iterable.

If you set strict to True, then you’ll get a ValueError exception when the input iterables don’t have the same length:

Python
>>> list(zip(range(5), range(100)))
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)]

>>> list(zip(range(5), range(100), strict=True))
Traceback (most recent call last):
    ...
ValueError: zip() argument 2 is longer than argument 1

In the first argument, you rely on the default value of strict and get five tuples because the shortest range only has five values. In the second example, you set strict to True. This time, you get an error because the input ranges don’t have the same number of values.

Building and Consuming Iterators: iter() and next()

In Python, iterators implement the iterator design pattern, which allows you to traverse a container and access its elements. The iterator pattern decouples the iteration algorithms from containers, such as lists, tuples, dictionaries, and sets.

Python has two built-in functions that can help you out when working with iterators. The iter() function allows you to build an iterator from an iterable, and the next() function lets you consume an iterator one item at a time.

Consider the following toy example:

Python
>>> colors = ["red", "orange", "yellow", "green"]

>>> colors_it = iter(colors)
>>> colors_it
<list_iterator object at 0x10566a170>

>>> next(colors_it)
'red'
>>> next(colors_it)
'orange'
>>> next(colors_it)
'yellow'
>>> next(colors_it)
'green'
>>> next(colors_it)
Traceback (most recent call last):
    ...
StopIteration

In this example, you use the iter() function to create an iterator object from an existing list of colors. Unlike iterables, iterators support the next() function. When you call this function with an iterator as an argument, you get the first item in the first call. When you call the function again, you get the second item, and so on.

When you traverse all the items in the iterator, next() raises a StopIteration exception. Python internally uses this exception to stop the iteration process in a for loop or a comprehension. Note that you can traverse an iterator only once. After that, the iterator will be exhausted or consumed.

Here are the signatures of iter() :

Python
iter(iterable)
iter(object, sentinel)

In the first signature, iterable represents any iterable type. In the second signature, the object argument must be a callable. You’ve already seen the first signature in action. Now, it’s time to take a quick look at the second signature.

To illustrate with an example, say that you’re working on a command-line interface (CLI) app and want to take the user’s input until they enter the word "done". Here’s how you can do this using the iter() function:

Python
>>> def read_user_input():
...     print("Enter word (type 'done' to finish):")
...     for word in iter(input, "done"):
...         print(f"Processing word: '{word}'")
...

>>> read_user_input()
Enter word (type 'done' to finish):
Python
Processing word: 'Python'
Programming
Processing word: 'Programming'
Iterators
Processing word: 'Iterators'
done

In the highlighted line, you use iter() with two arguments. For the first argument, you use the built-in input() function, which allows you to take input from the user. Note that you don’t call the function but pass it as a function object.

Then, you have the word "done", which works as a sentinel. In other words, iter() will call input() for you and raise a StopIteration exception if its return value matches the sentinel word.

When you call the function, you’re asked to enter a word, then the code processes the word and lets you enter another word. These actions repeat until you enter your sentinel, the word "done".

When it comes to the next() function, you’ll also have two different signatures that looks something like this:

Python
next(iterator)
next(iterator, default)

Again, you’ve seen how to use next() with an iterable as its unique argument. Now, you can focus on using the second signature. In this case, you have a second argument called default. This argument allows you to provide the value that you want to return when the input iterable is exhausted or its data is over:

Python
>>> count_down = iter([3, 2, 1])

>>> next(count_down, 0)
3
>>> next(count_down, 0)
2
>>> next(count_down, 0)
1
>>> next(count_down, 0)
0
>>> next(count_down, 0)
0

In this example, you create an iterator from a list of numbers. Then, you use next() to consume the iterator one number at a time. After next() has consumed the entire iterator, you get 0 as a result because that’s the value you passed to default, the second positional argument. Successive calls to the function will return 0 as well.

Filtering and Mapping Iterables: filter() and map()

Have you heard about functional programming? It’s a programming paradigm where a program is dominated by calls to pure functions, which are functions whose output values depend solely on their input values without any observable side effects.

Python isn’t what you could call a functional programming language. However, it has a couple of built-in functions that are classical functional tools. These tools are the built-in filter() and map() functions.

You can use the filter() function to extract values from iterables, which is known as a filtering operation.

The signature of filter() looks something like this:

Python
filter(function, iterable)

The first argument, function, must be a single-argument function, while the second argument can be any Python iterable. Here’s a short description of these arguments:

Argument Description
function A predicate or Boolean-valued function that accepts a single argument
iterable A Python iterable

The function argument is a decision function, also known as a filtering function. It provides the criteria for deciding whether to keep a given value.

In practice, filter() applies function to all the items in iterable. Then, it creates an iterator that yields only the items that meet the criteria that function checks. In other words, it yields the items that make function return True.

The filter() function allows you to process iterables without a formal loop. Here’s an example of using filter() to extract even numbers from a list of values:

Python
>>> numbers = [1, 3, 10, 45, 6, 50]

>>> list(filter(lambda n: n % 2 == 0, numbers))
[10, 6, 50]

In this example, the lambda function takes an integer and returns True if the input value is an even number and False otherwise. The call to filter() applies this lambda function to the values in numbers and filters out the odd numbers, returning the even numbers. Note that you use the list() function to create a list from the iterator that filter() returns.

The map() function is another common tool in functional programming. This function allows you to apply a transformation function to all the values in an iterable.

Python’s map() has the following signature:

Python
map(function, iterable, *iterables)

The map() function applies function to each item in iterable in a loop and returns a new iterator that yields transformed items on demand.

Here’s a summary of the arguments and their meanings:

Argument Description
function A Python function that takes a number of arguments equal to the number of input. iterables
iterable A required argument that can hold any Python iterable.
*iterables A variable number of Python iterables.

The function argument is what is called a transformation function. It applies a specific transformation to its arguments and returns a transformed value.

To illustrate how map() works, say that you have two lists. The first list contains a series of values that you want to use as the bases in power computations. The second list contains the exponents that you want to apply to each base. You can use the map() function to process these lists and get an iterator of powers:

Python
>>> bases = [8, 5, 2]
>>> exponents = [2, 3, 4]

>>> list(map(pow, bases, exponents))
[64, 125, 16]

In this example, you use the built-in pow() function as the first argument to map(). As you already learned, pow() takes a base and an exponent as arguments and returns the power. Then, you pass the bases and exponents to map() so that it computes the desired powers.

Processing Input and Output

If you need to take input from the user or files and present output to the user, then you should know that the language has a few built-in functions that can help you with these tasks:

Function Description
input() Reads input from the console
open() Opens a file and provides access to a file object
print() Prints to a text stream or the console
format() Converts a value to a formatted representation

In the following sections, you’ll dive into using these functions to process input and output operations in your Python code.

Accepting Input From the User: input()

Taking input from your users is a common operation in CLI and text-based interface (TUI) applications. Python has a built-in function that’s specifically targeted to this type of operation. The function is conveniently called input().

The built-in input() function reads the user’s input and grabs it as a string. Here’s the function’s signature:

Python
input([prompt])

The square brackets around prompt are an indication that this argument is optional. This argument allows you to provide a prompt to ask the user for the required or desired input.

As an example of using input(), say that you want to create a number-guessing game. The game will prompt the user to enter a number from 1 to 10 and check if the input value matches a secret number.

Here’s the code for the game:

Python guess.py
from random import randint

LOW, HIGH = 1, 10

secret_number = randint(LOW, HIGH)
clue = ""

while True:
    guess = input(f"Guess a number between {LOW} and {HIGH} {clue} ")
    number = int(guess)
    if number > secret_number:
        clue = f"(less than {number})"
    elif number < secret_number:
        clue = f"(greater than {number})"
    else:
        break

print(f"You guessed it! The secret number is {number}")

In this code, you define an infinite loop where you prompt the user to make their guess by entering a number between 1 and 10. The first line in the loop is a call to the built-in input() function. You’ve used a descriptive prompt to inform the users what to do.

Go ahead and run the script from your command line to try it out:

Shell
$ python guess.py
Guess a number between 1 and 10  2
Guess a number between 1 and 10 (greater than 2) 3
Guess a number between 1 and 10 (greater than 3) 8
Guess a number between 1 and 10 (less than 8) 6
You guessed it! The secret number is 6

Cool! Your game asks the user to enter a number, compares it with the secret number, and lets the users know when they guess correctly. The input() function plays a core role in the game’s flow, allowing you to get and process the user’s input.

Opening Files: open()

Reading from and writing to files are common programming tasks. In Python, you can use the built-in open() function for these purposes. You typically use the open() function in a with statement.

As a quick example, say that you have a text file with the following content:

Text fruits.txt
apple
banana
cherry
orange
mango

You want to open the file and read its content while you print it to the screen. To do this, you can use the following code:

Python
>>> with open("fruits.txt") as file:
...     print(file.read())
...
apple
banana
cherry
orange
mango

In this with statement, you call open() with the filename as an argument. This call opens the file for reading. The open() function returns a file object, which the with statement assigns to the file variables with the as specifier.

The open() function has the following signature:

Python
open(
    file,
    mode="r",
    buffering=-1,
    encoding=None,
    errors=None,
    newline=None,
    closefd=True,
    opener=None,
)

The function can take up to eight arguments. The first argument, file, is the only required argument. The rest of the arguments are optional. Here’s a summary of the arguments’ meaning:

Argument Description Comment
file A path-like object holding the path to the target file It’s a required argument.
mode A string that specifies the mode in which you want to open the file It defaults to "r", which is the reading mode. You’ll learn about the available modes in a moment.
buffering An integer that sets the buffering policy You can pass 0 to switch buffering off, which is only possible in binary mode. You can use 1 to select line buffering, which is only usable in text mode. Finally, you can use an integer greater than 1 to indicate the size in bytes of a fixed-size chunk buffer.
encoding The name of the encoding used to decode or encode the file You can only use this argument in text mode.
errors A string that specifies how encoding and decoding errors are to be handled You can only use this argument in text mode. It can take one of the following values: "strict", "ignore", "replace", "surrogateescape", "xmlcharrefreplace", "backslashreplace", or "namereplace". These values have similar meanings to those you learned in the section about the ord() and chr() functions.
newline A string that determines how to parse newline characters from the stream It can be None, "", "\n", "\r", or "\r\n".
closefd A Boolean value that defines whether you want to close a file descriptor It can be False when you provide a file descriptor instead of a filename and want the descriptor to remain open when the file is closed. Otherwise, it must be True.
opener A callable that you use as a custom opener for the target file The opener must return an open file descriptor.

In this tutorial, you won’t cover all these arguments. Instead, you’ll learn about two of the most commonly used arguments, which are mode and encoding.

Here’s a list of allowed mode values:

Value Description
"r" Opens the file for reading and is the default value
"w" Opens the file for writing, truncating the file first
"x" Opens the file for exclusive creation, failing if the file already exists
"a" Opens the file for writing, appending the new data to the end of the file if it already exists
"b" Opens the file in binary mode
"t" Opens the file in text mode, which is the default mode
"+" Opens the file for updating, which allows reading and writing operations

In this table, the "b" and "t" values define two generic modes for binary and text files, respectively. You can combine these two modes with other modes. For example, the "wb" mode allows you to write binary data to a file, the "rt" mode enables you to read text-based data from a file, and so on.

Note that "t" is the default mode. So, if you set the mode to "w", Python assumes that you want to write text to the target file.

Here’s a code snippet that writes some text to a file in your working directory:

Python
>>> with open("hello.txt", "w") as file:
...     file.write("Hello, World!")
...
13

In this example, you open a file called hello.txt so you can write text to it. In the code block of the with statement, you call the .write() method on the file object to write some text. Note that the method returns the number of written bytes. That’s why you get 13 on the screen.

After running the code, you’ll have the hello.txt file in your working directory. Go ahead and open it to check its content.

You can experiment with other modes and get an idea of how they work so that you can use them safely in your code. Take this as a practical exercise!

Using the encoding argument is another typical requirement when you’re working with text files. In this situation, it’s a best practice to explicitly state the text encoding that you’re using in your code. The UTF-8 encoding is a common example of a value that you’d pass to encoding:

Python
>>> with open("hello.txt", "w", encoding="utf-8") as file:
...     file.write("Hello, Pythonista!")
...
13

>>> with open("hello.txt", "r", encoding="utf-8") as file:
...     print(file.read())
...
Hello, Pythonista!

In this example, you use the UTF-8 encoding to write and read from a text file. Note that you need to explicitly use the argument’s name to provide the encoding value. This is because the following argument in the list is buffering rather than encoding, and if you don’t use the explicit name, then you get a TypeError exception.

Printing Text to the Screen or Another Output: print()

Another common requirement that arises when you’re creating CLI or TUI applications is to display information on the screen to inform the user about the app’s state. In this case, you can use the built-in print() function, which is a fundamental tool in Python programming.

The print() function has the following signature:

Python
print(*objects, sep=" ", end="\n", file=None, flush=False)

Calling print() will print the input objects to the screen by default. You can use the rest of the arguments to tweak how the function works. Here’s a summary of the arguments and their meaning:

Argument Description
*objects An arbitrary number of Python objects
sep The string you want to use to separate input objects from one another
end The string to use after the last input object
file The open file object where you want to write the input objects
flush A Boolean value that defines whether you want to flush the output buffer

When you call print(), it takes the input objects, converts them into strings, joins them using sep, and appends end. If you call print() with no argument, then it prints end. Note that the arguments sep, end, file, and flush are keyword arguments.

Here are a few examples of how to use the print() function:

Python
>>> print()

>>> print("Hello")
Hello
>>> print("Hello", "Pythonista!")
Hello Pythonista!
>>> print("Hello", "Pythonista!", sep="\t")
Hello    Pythonista!
>>> print("Hello", "Pythonista!", sep="\t", end=" 👋\n")
Hello    Pythonista! 👋

When you call print() with no arguments, then end is printed on the screen. This argument defaults to a newline character, so that’s what you get. With an object as an argument, the object is printed, followed by a newline character. With several objects as arguments, the arguments are joined by sep, and a newline is added at the end.

You can also tweak the value of end and make Python print something different at the end of your output.

The file argument defaults to the standard output, which is your screen. The sys.stdout stream provides this default value. However, you can redirect the output to a file object of your preference:

Python
>>> with open("hello.txt", mode="w") as text_file:
...     print("Hello, World!", file=text_file)
...

This code snippet will override your existing hello.txt file from the previous section, writing the phrase "Hello, World!" into it.

Finally, you have the flush argument that has to do with data buffering. By default, Python buffers the calls to print() in a RAM data buffer. This allows Python to make fewer system calls for write operations by batching characters in the buffer and writing them all at once with a single system call.

You can set the flush argument to True if you want your code’s output to display in real time. If you keep flush at its default value of False, then Python will buffer the output, and that output will only show up once the data buffer is full or when your program finishes execution.

A cool example of using flush is when you need to create a progress bar for a CLI application. Consider the following function:

Python progress.py
def progress(percent=0, width=30):
    end = "" if percent < 100 else "\n"
    left = width * percent // 100
    right = width - left
    print(
        "\r[",
        "#" * left,
        " " * right,
        "]",
        f" {percent:.0f}%",
        sep="",
        end=end,
        flush=True,
    )

This function generates a horizontal progress bar by taking advantage of the flush argument. Here’s how you can use it in your code:

Python
>>> from time import sleep
>>> from progress import progress

>>> for percent in range(101):
...     sleep(0.2)
...     progress(percent)
...
[###########                   ] 38%

This loop calls progress() with successive hypothetical progress values. The output of each call is flushed and the progress bar is displayed in the same line.

Formatting Strings: format()

Python has a couple of handy tools for string interpolation and formatting, including f-strings and the str.format() method. These tools take advantage of Python’s string formatting mini-language, which allows you to nicely format your strings using a dedicated syntax.

The built-in format() function is another tool that you can use to format values. The function has the following signature:

Python
format(value, format_spec="")

The function converts value to a formatted representation. To define the desired format, you can use the format_spec argument, which accepts a string that follows the syntax defined in the string formatting mini-language. The format_spec argument defaults to an empty string, which causes the function to return the value as passed.

Consider the following examples of using the format() function:

Python
>>> import math
>>> from datetime import datetime

>>> format(math.pi, ".4f")  # Four decimal places
'3.1416'

>>> format(math.pi, "e")  # In scientific notation
'3.141593e+00'

>>> format(1000000, ",.2f")  # Thousand separators
'1,000,000.00'

>>> format("Header", "=^30")  # Centered and filled
'============Header============'

>>> format(datetime.now(), "%a %b %d, %Y")  # Date
'Mon Jul 1, 2024'

In these examples, you’ve used several different format specifiers. The ".4f" specifier formats the input value as a floating-point number with four decimal places. The "e" specifier allows you to format the input value using scientific notation.

With the ",.2f" format specifier, you can format a number using commas as thousand separators and with two decimal places, which is an appropriate format for currency values. Then, you use the "=^30" specifier to format the string "Header" centered in a width of 30 characters using the equal sign as a filler character. Finally, you use the "%a %b %d, %Y" to format a date.

Working With Classes, Objects, and Attributes

Python supports object-oriented programming (OOP) with classes, types, inheritance, and many other related features. In Python, everything is an object. So, the OOP paradigm is core to the language itself.

You’ll have several built-in functions that help you with different tasks related to classes, types, attributes, methods, inheritance, and other OOP-related concepts.

Here’s a summary of Python’s OOP-related built-in functions:

Function Description
property() Returns a property value of a class
classmethod() Returns a class method
staticmethod() Returns a static method
getattr() Returns the value of a named attribute of an object
setattr() Sets the value of a named attribute of an object
delattr() Deletes an attribute from an object
hasattr() Returns True if an object has a given attribute
type() Return the type of an object or allows for creating new classes dynamically
isinstance() Determines whether an object is an instance of a given class
issubclass() Determines whether a class is a subclass of a given class
callable() Returns True if an object appears to be callable
super() Returns a proxy object that delegates method calls to a parent or sibling class
object() Creates a new featureless object

In the following sections, you’ll learn about all these functions and how to use them in your object-oriented Python code.

Building Properties: property()

Python’s built-in property() function allows you to create managed attributes in your classes. Managed attributes, also known as properties, have an associated value and an internal implementation or function-like behavior.

To illustrate this with an example, say that you want to create a Point class. In Python, you’ll start with something like the following:

Python
>>> class Point:
...     def __init__(self, x, y):
...         self.x = x
...         self.y = y
...

>>> point = Point(42, 21)

>>> point.x
42
>>> point.y
21

>>> point.x = 0
>>> point.x
0

In this class, you define two attributes, .x and .y, to represent the point’s coordinates. You can access and update the attributes directly using the dot notation. So, from now on, both attributes are part of your class’s public API.

Now, say that you need to add some validation logic on top of .x and .y. For example, you may need to validate the input values for both attributes. How would you do that? In programming languages like Java or C++, you’d use the getter and setter methods, which translated into Python may look something like this:

Python point_v1.py
class Point:
    def __init__(self, x, y):
        self.set_x(x)
        self.set_y(y)

    def get_x(self):
        return self._x

    def set_x(self, x):
        self._x = self.validate(x)

    def get_y(self):
        return self._y

    def set_y(self, y):
        self._y = self.validate(y)

    def validate(self, value):
        if not isinstance(value, int | float):
            raise ValueError("coordinates must be numbers")
        return value

In this new implementation of Point, you’ve turned .x and .y into non-public attributes by prepending underscores to their names, which now are ._x and ._y. Then, you define getter and setter methods for both attributes. In the setter methods, .set_x() and .set_y(), you insert the validation logic defined in the .validate() method.

Now, you have to use the class as in the following code:

Python
>>> from point_v1 import Point

>>> point = Point(42, 21)

>>> point.get_x()
42
>>> point.get_y()
21

>>> point.set_x(0)
>>> point.get_x()
0
>>> point.set_y("7")
Traceback (most recent call last):
    ...
ValueError: coordinates must be numbers

Your class works differently after the update. Instead of accessing the .x and .y attributes directly, you have to use the getter and setter methods. The validation logic works, which is great, but you’ve broken your class’s API. Your users won’t be able to do something like the following:

Python
>>> point.x
Traceback (most recent call last):
    ...
AttributeError: 'Point' object has no attribute 'x'

Old users of your class will be surprised that their code is broken after updating to your new version of Point. So, how can you avoid this kind of issue? The Pythonic approach is to convert public attributes into properties instead of using getter and setter methods.

You can use the built-in property() function to do this conversion. Here’s how you can keep your Point class’s API unchanged:

Python point_v2.py
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @property
    def x(self):
        return self._x

    @x.setter
    def x(self, value):
        self._x = self.validate(value)

    @property
    def y(self):
        return self._y

    @y.setter
    def y(self, value):
        self._y = self.validate(value)

    def validate(self, value):
        if not isinstance(value, int | float):
            raise ValueError("coordinates must be numbers")
        return value

Point now looks a bit different. It doesn’t have formal getter and setter methods. Instead, it has some methods that are decorated with @property. Yes, the built-in property() function is mostly used as a decorator.

The methods that you decorate with @property are equivalent to getter methods. Meanwhile, the methods that you decorate with the getter’s name plus .setter() are equivalent to setter methods. The cool thing about properties is that you can still use the attributes as regular attributes:

Python
>>> from point_v2 import Point

>>> point = Point(42, 21)

>>> point.x
42
>>> point.y
21

>>> point.x = 0
>>> point.x
0

>>> point.x = "7"
Traceback (most recent call last):
    ...
ValueError: coordinates must be numbers

By turning regular attributes into properties, you can add function-like behavior to them without losing the ability to use them as regular attributes. Properties save you from introducing breaking changes into your code’s public API, so you don’t break your users’ code.

Creating Class and Static Methods: classmethod() and staticmethod()

Classes allow you to define reusable pieces of code that encapsulate data and behavior in a single entity. Usually, you store data in attributes, which are variables defined inside classes. When it comes to behaviors, you’ll use methods, which are functions defined in classes.

In Python, you have three different types of methods:

  1. Instance methods, which take the current object as their first argument
  2. Class methods, which take the current class as their first argument
  3. Static methods, which take neither the current instance nor the current class as arguments

Instance methods need to take the current instance as an argument. By convention, this argument is called self in Python.

To create a class method, you need to decorate the method with the @classmethod decorator. Similarly, to create a static method, you need to decorate the method with the @staticmethod decorator. Both decorators are part of Python’s built-in functions.

A common use case for class methods is to provide multiple constructors for a class. To illustrate how to write a class method, say that you want a Point class that you can construct using either Cartesian or polar coordinates. In this situation, you can do something like the following:

Python point.py
import math

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @classmethod
    def from_polar(cls, distance, angle):
        return cls(
            x=distance * math.cos(math.radians(angle)),
            y=distance * math.sin(math.radians(angle)),
        )

In this example, the .from_polar() method is a class method. It takes the current class as its first argument, which you typically call cls by convention. The method returns a new instance of the class by computing the Cartesian coordinates from the polar coordinates.

Here’s how you can use this method in practice:

Python
>>> from point import Point

>>> point = Point.from_polar(20, 15)

>>> point.y
5.176380902050415
>>> point.x
19.318516525781366

In this code snippet, you create a new Point instance using the .from_polar() class method. In the example, you call the method on the class rather than on an instance to signal that this is a class method. You can call a class method on an instance of its containing class too.

The third type of method is the static method. A static method doesn’t take the current instance or class as arguments. These methods are like regular functions that you decide to include in a given class for convenience. Functionally, they could also be defined as regular functions in a module.

For example, consider the following Formatter class:

Python formatting.py
class Formatter:
    @staticmethod
    def as_currency(value):
        return f"${value:,.2f}"

    @staticmethod
    def as_percent(value):
        return f"{value:.2%}"

This class defines two static methods. The first method takes a numeric value and formats it as a currency value. The second method takes a numeric value and expresses it as a percent. You could have defined these methods as regular functions at the module level. However, you’ve defined them in a class as a way to conveniently group them according to how they’ll be used.

You can use this class as in the following examples:

Python
>>> from formatting import Formatter

>>> Formatter.as_currency(1000)
'$1,000.00'

>>> Formatter.as_percent(0.75)
'75.00%'

>>> formatter = Formatter()
>>> formatter.as_currency(1000)
'$1,000.00'

>>> formatter.as_percent(0.8)
'80.00%'

You can use static methods by calling them on the class or one of its instances. In this example, the Formatter class works as a namespace where you define related methods for convenience. However, you can get the same result by defining the methods as module-level functions.

Managing Attributes: getattr(), setattr(), and delattr()

Sometimes, you may need to access, set, or delete attributes from your objects in Python. In most cases, you can do these operations directly using the dot notation, the assignment operator, and the del statement.

In other situations, you only know the attributes’ names at runtime, so you can’t access them with the regular syntax. In these cases, you can use the built-in getattr(), setattr(), and delattr() functions. These functions have the following signatures:

Python
getattr(object, name)
getattr(object, name, default)

setattr(object, name, value)

delattr(object, name)

In all cases, the object argument must take an instance of an existing class. Similarly, name must be the name of an attribute or method as a string.

In the second signature of getattr(), the default argument is an optional value that you’ll get if the desired attribute doesn’t exist in the target object.

In the signature of setattr(), the value argument must hold the new value that you want to assign to a given argument.

To illustrate how these functions work, consider the following class:

Python person.py
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

This class has two instance attributes, .name and .age. Here’s how you can access, set, or delete the attributes using their names as strings:

Python
>>> from person import Person

>>> jane = Person("Jane", 25)

>>> getattr(jane, "name")
'Jane'
>>> getattr(jane, "age")
25

In these examples, you use getattr() to retrieve the values stored in .name and .age. The first argument to this function is the object you need to retrieve an attribute from. The second argument is the attribute’s name as a string.

Now say that you want to update Jane’s age. You can do this using the setattr() function:

Python
>>> setattr(jane, "age", 26)
>>> jane.age
26

Then, you use the setattr() function to assign a new value to the .age attribute. This function takes three arguments: the object, the attribute’s name, and the new value.

Finally, you can use the built-in delattr() function to delete an attribute from a given object:

Python
>>> delattr(jane, "age")
>>> jane.age
Traceback (most recent call last):
    ...
AttributeError: 'Person' object has no attribute 'age'

The delattr() function takes the object as its first argument and the attribute’s name as its second argument. After calling the function, trying to access .age will raise an AttributeError exception.

In practice, the built-in getattr(), setattr(), and delattr() functions come in handy when you need to manipulate attributes using their names as strings. For example, say that you want to create a FileProcessor class to read and write CSV and JSON files. In this situation, you can have dedicated classes for processing each file type:

Python processors.py
import csv
import json

class CSVProcessor:
    def __init__(self, filename):
        self.filename = filename

    def read(self):
        with open(self.filename, encoding="utf-8", newline="") as file:
            return list(csv.DictReader(file))

    def write(self, data):
        with open(
            self.filename, mode="w", encoding="utf-8", newline=""
        ) as file:
            writer = csv.DictWriter(file, fieldnames=data[0].keys())
            writer.writeheader()
            writer.writerows(data)

class JSONProcessor:
    def __init__(self, filename):
        self.filename = filename

    def read(self):
        with open(self.filename, encoding="utf-8") as file:
            return json.load(file)

    def write(self, data):
        with open(self.filename, mode="w", encoding="utf-8") as file:
            json.dump(data, file, indent=2)

In this processors.py file, you define two classes that can process CSV and JSON files, respectively. Both classes have the .read() and .write() methods. These classes look fine, but now you need to make them usable from your FileProcessor class.

To write the FileProcessor class, you can use a technique called delegation, which consists of evaluating an object’s attribute or method in the context of another object. Here’s how you can do this in Python:

Python processors.py
# ...

class FileProcessor:
    def __init__(self, filename, processor):
        self.filename = filename
        self.processor = processor(filename)

    def __getattr__(self, attr):
        return getattr(self.processor, attr)

In this class, you define the .__getattr__() special method. This method supports attribute access operations in Python classes. In the method definition, you use the getattr() function to access attributes and methods from the provided processor object.

In practice, you use the combination of the .__getattr__() method and the getattr() function to implement delegation. The FileProcessor class delegates the file processing to the concrete processor class that you pass in during instantiation.

Here’s how you can use the FileProcessor class in your code:

Python
>>> from processors import FileProcessor

>>> file_proc = FileProcessor("products.csv", CSVProcessor)
>>> file_proc.read()
[
    {'product': 'Laptop', 'price': '1200', 'sold_units': '30'},
    {'product': 'Phone', 'price': '700', 'sold_units': '50'},
    {'product': 'Tablet', 'price': '450', 'sold_units': '100'},
    {'product': 'Desktop', 'price': '1000', 'sold_units': '20'},
    {'product': 'Monitor', 'price': '300', 'sold_units': '50'}
]

In this code, you create a FileProcessor instance to process a CSV file with the CSVProcessor. Even though the instance doesn’t have a .read() method, you can call the method because of the delegation technique that relies on the getattr() function.

Checking for Attributes: hasattr()

Another built-in function closely related to attributes and methods is the hasattr() function. This function allows you to check whether a given object has a certain attribute or method. The function has the following signature:

Python
hasattr(object, name)

In this signature, the object argument can take any Python object, while the name argument should hold the name of an attribute as a string. This function is a predicate that returns True if the object has an attribute with the provided name and False otherwise.

In practice, you can use this function to check whether an object has a given attribute or method before you try to use it. For example, say that you have the following classes:

Python birds.py
class Duck:
    def fly(self):
        print("The duck is flying")

    def swim(self):
        print("The duck is swimming")

class Pigeon:
    def fly(self):
        print("The pigeon is flying")

These classes represent two different birds. Both birds are capable of flying, but only the duck is capable of swimming. Now, say that you want to use them in a loop like the following:

Python
>>> from birds import Duck, Pigeon

>>> birds = [Duck(), Pigeon()]

>>> for bird in birds:
...     bird.fly()
...     bird.swim()
...
The duck is flying
The duck is swimming
The pigeon is flying
Traceback (most recent call last):
    ...
AttributeError: 'Pigeon' object has no attribute 'swim'

This loop works for the instance of Duck. However, it raises an AttributeError exception when you call .swim() on a Pigeon instance because the class doesn’t have this method. To avoid this error, you can use the hasattr() function to check whether the method exists before calling it:

Python
>>> for bird in birds:
...     bird.fly()
...     if hasattr(bird, "swim"):
...         bird.swim()
...
The duck is flying
The duck is swimming
The pigeon is flying

Your code doesn’t fail now because you’ve used the hasattr() function to ensure that the current bird has the .swim() method before calling it.

Creating and Checking Types: type(), isinstance() and issubclass()

Python is a dynamically typed language, which means that Python checks types only as the code runs, and the type of a variable can change over its lifetime. Because of this language feature, you may need to explicitly check an object’s type before using it so that your code doesn’t fail.

To know the type of a given object, you can use the built-in type() function:

Python
>>> type(42)
<class 'int'>

>>> type(2.75)
<class 'float'>

>>> type("Hello")
<class 'str'>

When you call type() with any Python class as an argument, then you get the object’s type, which you can also call the object’s class. In this example, you call type() with an integer number as an argument and get the int class as a response. Then, you use type() with a floating-point number and get the float class, and so on.

If you want to check an object’s type with type(), then you can do something like the following:

Python
>>> type(42) == int
True
>>> type(42) == float
False

This way of using type() works. However, it’s not the recommended approach. You’ll learn more about type checking in a moment. For now, you’ll continue to learn the basics of type(). To kick things off, here are the function’s signatures:

Python
type(object)
type(name, bases, dict, **kwds)

You’ve already used the first signature. In this signature, the object argument represents any Python object.

The second signature is a bit more involved. You’ll use this signature to create new classes dynamically rather than to determine an object’s type. Here’s a summary of the arguments and their meaning:

Argument Description
name The class’s name
base A tuple containing the base classes
dict A dictionary of attributes and methods defined in the class body
**kwds Additional keyword arguments that are passed to the metaclass constructor

When you use type() with these arguments, you can build classes dynamically. In this way, type() is a dynamic form of the class statement. Consider the following toy example:

Python
>>> def greet(self):
...     print("Hello, World!")
...

>>> DemoClass = type("DemoClass", (), {"value": 42, "greet": greet})
>>> DemoClass.value
42
>>> instance = DemoClass()
>>> instance.value
42
>>> instance.greet()
Hello, World!

>>> dir(instance)
[
    '__class__',
    '__delattr__',
    '__dict__',
    ...
    'greet',
    'value'
]

In this quick example, you use type() to create a demo class that automatically inherits from object because the base tuple is empty. The new class will have a method called .greet(), which you’ve defined beforehand. It also has a class attribute called .value that you set to 42.

For attributes, you should provide the attribute’s name as a string and the attribute’s value. For methods, you should give the method’s name as a string and a method object, which is a method without the calling parentheses. Note that instance methods like .greet() must take the current object as an argument, which you typically call self.

For a more realistic example, say that you want to write a function that lets you build classes dynamically from different data schemas. In this situation, you can do something like the following:

Python factory.py
def create_class(name, custom_members):
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)

    def __repr__(self):
        return f"{name}({self.__dict__})"

    class_members = {
        "__init__": __init__,
        "__repr__": __repr__,
    }
    class_members.update(custom_members)

    return type(name, (), class_members)

In this code, you create a function that takes two arguments. The first argument, name, should be a string providing a valid class name. The second argument, custom_members, should be a dictionary of attributes and methods.

Then, you define an inner function called .__init__(), which you’ll use as the class initializer. The .__repr__() function will allow you to provide a string representation for the objects of your class.

Next, you create a dictionary to include the functions as methods for your class and update the dictionary with the content of class_members, which should come from the user.

Finally, you use the type() function to generate the class with the provided name and the dictionary of members. Here are a couple of examples of how to use this function:

Python
>>> from factory import create_class

>>> User = create_class("User", {"name": "", "age": 0, "email": ""})
>>> Product = create_class(
...     "Product", {"name": "", "price": 0.0, "units": 0}
... )

>>> john = User(name="John", age=30, email="john@example.com")
>>> table = Product(name="Table", price=200.0, units=5)

>>> john.name
'John'
>>> john.age
30
>>> john.email
'john@example.com'

>>> table.name
'Table'
>>> table.price
200.0
>>> table.units
5

In this code snippet, you first create two classes using create_class(). The first class represents users, and the second represents products. Both have different sets of instance attributes.

Then, you create concrete instances of each class with proper values for the attributes. Finally, you access the attributes using the dot notation. That’s great! Your classes work as expected.

The type() function is a great tool for creating classes dynamically. Even though you can also use this function to check for an object’s type, the recommended tool for explicit type checking is the built-in isinstance() function because it takes subclasses into account.

The signature for isinstance() is like the following:

Python
isinstance(object, classinfo)

In this signature, object represents any Python object in which you’re interested. The classinfo argument is the class or classes that you want to check against. This argument can be a single class object, a tuple of class objects, or a union type.

Consider the following examples where you use isinstance() to check for numeric values:

Python
>>> isinstance(42, int)
True

>>> isinstance(42.0, (int, float))
True

>>> isinstance(42.0, int | float)
True

In the first example, you use isinstance() to check whether 42 is an instance of the int class. In the second example, you use isinstance() to check whether 42.0 is an instance of either int or float. In this example, you use a tuple of classes to provide the classinfo argument.

Finally, in the third example, you do the same check as in the second example. This time, you use the pipe character (|) to create a union type with int and float. Note that isinstance() is a predicate function that returns True if the input object is an instance of one of the provided classes.

The isinstance() function also considers subclasses. For example, the bool class is a subclass of int so if you compare an instance of bool with int, then you’ll get True as a result:

Python
>>> isinstance(False, int)
True

>>> type(False) == int
False

Because bool is a subclass of int, the isinstance() function returns True when you check a Boolean value against the int class. Note that if you try to do a similar check with type(), then you get False because type() doesn’t consider subclasses.

There’s another built-in function that can be useful for type checking. The function is called issubclass() and it checks whether a given class is a subclass of another class:

Python
>>> issubclass(int, object)
True

>>> issubclass(bool, int)
True

>>> issubclass(int, float)
False

In the first example, you check whether the int class is a subclass of object. In this case, you get True because all Python classes derive from object. Then, you check whether bool is a subclass of int, which is also true, as you already learned.

In the final example, you use issubclass() to check whether int is a subclass of float, which is False.

The signature of issubclass() is as below:

Python
issubclass(class, classinfo)

In this case, the class argument is the class that you want to check for, while the classinfo argument works the same as in isinstance().

Checking for Callable Objects: callable()

A callable in Python is any object that you can call using a pair of parentheses and a series of arguments if required. In Python, callable objects include functions, classes, methods, instances of classes with a .__call__() method, closures, and generator functions.

Sometimes, you may need to know whether an object is callable before calling it in your code. To do this, you can use the built-in callable() function, which takes an object as an argument and returns True if the object appears to be callable. Otherwise, it returns False.

Here are a few examples of using callable() with some built-in objects:

Python
>>> callable(abs)
True
>>> callable(int)
True
>>> callable(list)
True

>>> callable(True)
False
>>> callable(None)
False

In the first three examples, the arguments to callable() are all functions, so you get True as a result. In the final two examples, you use the True and None objects as arguments. These objects aren’t callable, so you get False as a result.

As a practical example, say that you need to build an app that processes commands. Every command should be callable, otherwise it won’t be valid. To check this condition, you can use callable(). Here’s a toy implementation:

Python commands.py
class CommandProcessor:
    def __init__(self):
        self.commands = {}

    def register_command(self, command):
        if not callable(command):
            raise ValueError("command is not callable")
        self.commands[command.__name__] = command

    def execute_command(self, name, *args, **kwargs):
        if (command := self.commands.get(name)) is None:
            raise ValueError(f"command '{name}' not found")
        return command(*args, **kwargs)

In this class, the .register_command() method uses callable() to check whether the input command is a callable object. If that’s the case, then you register the command as valid. Next, you have the .execute_command() method that runs the command as a callable.

Here’s an example of how to use this class:

Python
>>> from commands import CommandProcessor

>>> command_processor = CommandProcessor()

>>> def add(a, b):
...     return a + b
...

>>> command_processor.register_command(add)
>>> command_processor.execute_command("add", 1, 2)
3

>>> subtract = 3 - 2
>>> command_processor.register_command(subtract)
Traceback (most recent call last):
    ...
ValueError: command is not callable

In this example, you create a CommandProcessor instance to process commands. Then, you write add() to use it as a command. Because add() is a callable object, you can register it as a valid command and run it with the .execute_command() method.

Finally, you define the subtract variable to hold the result of a subtraction operation. This variable isn’t callable. Therefore, you get a ValueError exception when registering it as a command.

Accessing the Parent’s Members: super()

When working with inheritance in Python classes, you’ll often need to access a parent class’s attributes or methods in a subclass. The Pythonic way to do this is to use the built-in super() function.

A common use case for super() is when you need to create a subclass of an existing class, and you need a proper way to initialize the parent class’s attributes. Consider the following classes that represent a rectangle and a square:

Python shapes.py
class Rectangle:
    def __init__(self, length, width):
        self.length = length
        self.width = width

    def area(self):
        return self.length * self.width

    def perimeter(self):
        return 2 * (self.length + self.width)

class Square(Rectangle):
    def __init__(self, side):
        super().__init__(side, side)

In this code, you define a Rectangle class with two attributes, .length and .width. It also has two methods for computing the rectangle’s area and perimeter. Next, you define the Square class. Since a square is a type of rectangle with equal sides, it makes sense to create Square as a subclass of Rectangle and reuse the functionality that you’ve already implemented.

In the Square() constructor, you only need a single argument to represent the side length. You can use this argument to initialize the parent class using super() as you did in the highlighted line. The super() function gives you access to the parent class Rectangle. Once you have access to the class, you can call its .__init__() method to initialize the .length and .width attribute with the value of side.

Building Generic Objects: object()

In Python, every single class implicitly inherits from the object class, which is built into the language. In other words, the object class is the base class for every class in Python:

Python
>>> issubclass(int, object)
True
>>> issubclass(float, object)
True
>>> issubclass(bool, object)
True
>>> issubclass(dict, object)
True

>>> class DemoClass:
...     pass
...
>>> issubclass(DemoClass, object)
True

No matter if the class you’re considering is a built-in or a custom class, it inherits from object.

In some situations, you may want to create instances of the object class. To do this, you can use the built-in object() function, which is actually a class constructor rather than a function, but the Python documentation lists it among its built-in functions.

The object() function doesn’t take an argument and returns a new featureless object, which has the methods that are common to all Python objects. Unlike regular objects, the object that you get from calling object() doesn’t have a .__dict__ attribute, so you can’t add attributes dynamically to this type of object:

Python
>>> obj = object()
>>> dir(obj)
[
    '__class__',
    '__delattr__',
    '__dir__',
    ...
    '__str__',
    '__subclasshook__'
]
>>> obj.attr = "Some value"
Traceback (most recent call last):
    ...
AttributeError: 'object' object has no attribute 'attr'

In this example, you create a new featureless object by calling object(). With the built-in dir() function, you can list all the methods and attributes that this object provides. Finally, if you try to add an attribute to your featureless object dynamically, then you get an AttributeError exception.

In practice, you can use the object() function when you want to create unique sentinel values. A sentinel value is a unique marker that you can use to signify the absence of a value. You can also use it as a condition for stopping iterative or recursive algorithms.

To illustrate how to use object() to create a sentinel value, consider the following Circle class:

Python circle.py
from time import sleep

SENTINEL = object()

class Circle:
    def __init__(self, radius):
        self.radius = radius
        self._diameter = SENTINEL

    @property
    def diameter(self):
        if self._diameter is SENTINEL:
            sleep(0.5)  # Simulate a costly computation
            self._diameter = self.radius * 2
        return self._diameter

In this class, you have a .radius attribute that takes a value at instantiation time. Then, you have a non-public attribute called ._diameter that you initialize with the SENTINEL constant. To create this constant, you use the object() function.

Finally, you have the .diameter property, which computes the diameter from the provided radius. In this example, you use sleep() from the time module to simulate that finding the diameter is a costly operation. Because of the computation cost, you decide to cache the diameter so that it’s computed a single time during the object’s life.

To check whether the diameter was already computed, you compare its current value with the SENTINEL constant. In this example, you could’ve also used None as the sentinel value because a circle’s diameter is unlikely to take a null value. However, when None can be a valid value for the attribute at hand, then object() can be the way to go.

Working With Python Scopes

Python, like many other programming languages, manages the concept of scopes. The scope rules how variables and names are looked up in your code. It determines the visibility of a variable or name within the code.

The scope depends on the place where you create that variable. Python’s scopes follow a rule known as the LEGB rule. The letters in this acronym stand for local, enclosing, global, and built-in scopes, and this rule summarizes the four scopes that you find in Python.

You’ll find two built-in functions that are closely related to scopes in Python. These functions are listed in the table below:

Function Description
locals() Updates and returns a dictionary representing the current local symbol table
globals() Returns a dictionary representing the current global symbol table

In the following sections, you’ll learn the basics of these functions and how to use them in your Python code to manage some aspects of your name scopes.

Inspecting and Updating a Local Scope: locals()

The local scope is the function scope because it comprises the function’s body. Every time you call a function, Python creates a new local scope for that function. By default, arguments and names that you assign in a function’s body exist only within the local scope that Python creates when you call the function. When the function returns, the local scope vanishes, and the names are forgotten.

If you ever need to inspect the state of your current local scope, then you can use the built-in locals() function:

Python
>>> def add(a, b):
...     result = a + b
...     print(locals())
...     return result
...

>>> add(2, 5)
{'a': 2, 'b': 5, 'result': 7}
7

In this function, you take two arguments, a and b. These arguments are local to add(), which means you can only access and use them inside the function. Then, you create a local variable called result, which you use as a temporary variable to store the computation’s results. The call to locals() returns a dictionary containing the names and values of all these variables.

Note that locals() grabs the scope information only to the point at which you call it:

Python
>>> def add(a, b):
...     print(locals())
...     result = a + b
...     return result
...

>>> add(2, 5)
{'a': 2, 'b': 5}
7

In this variation of add(), you call locals() at the beginning of the function. So, you only get the arguments in the result dictionary. This is because when you call locals(), the result variable hasn’t yet been defined.

Inspecting and Updating the Global Scope: globals()

The global scope is another important scope in Python. It’s the module-level scope and allows you to define global variables or names. You can access and modify global names from anywhere in your code.

To inspect and update the variables and names that live in your current global scope, you can use the built-in globals() function. For example, when you start a fresh REPL session and call globals(), then you get an output like the one below:

Python
>>> globals()
{
    '__name__': '__main__',
    '__doc__': None,
    ...
    '__builtins__': <module 'builtins' (built-in)>
}

By default, when you start a REPL session, the interpreter loads several names and objects to your global scope. For example, the __name__ object holds the current module’s name, which is "__main__" when you’re in an executable module. If you’re in an imported module, then this variable will hold the module’s name.

Then, you have the __doc__ name, which will hold the module’s docstring if provided. You’ll also have several other names. Finally, you have the __builtins__ name, which holds the namespace where built-in names are defined. This is a special module that includes all the built-in functions covered in this tutorial and several other built-in objects, like exceptions.

If you start defining variables and functions in your REPL session, then these names will be added to the dictionary that globals() returns:

Python
>>> language = "Python"
>>> number = 42
>>> def greet():
...     print("Hello, World!")
...
>>> class Demo:
...     pass
...

>>> globals()
{
    ...
    '__builtins__': <module 'builtins' (built-in)>,
    'language': 'Python',
    'number': 42,
    'greet': <function greet at 0x100984040>,
    'Demo': <class '__main__.Demo'>
}

In this example, you define two variables, a function, and a class. When you call globals(), you get the names of all those objects at the end of the resulting dictionary.

The dictionary that globals() returns is a writable dictionary. You can take advantage of this feature when you need to modify or update the global scope’s content manually. A common use case for this feature is when you need to load configuration parameters from a file.

For example, say that you have the following JSON file with some configuration values for your database connection:

JSON config.json
{
    "DATABASE_URL": "postgres://user:pass@localhost/dbname",
    "DEBUG_MODE": true,
    "MAX_CONNECTIONS": 10
}

You need to write a function that loads this file and adds the provided configuration parameters to your current global scope. Here’s a possible implementation of this function:

Python config.py
import json

def load_config(config_file):
    with open(config_file) as file:
        config = json.load(file)

    globals().update(config)

In this function, you first open the configuration file and load its content to a dictionary called config. Then, you update the dictionary that globals() returns with the content of config.

Here’s how the above function works:

Python
>>> from config import load_config

>>> load_config("config.json")
>>> globals()
{
    ...
    'DATABASE_URL': 'postgres://user:pass@localhost/dbname',
    'DEBUG_MODE': True,
    'MAX_CONNECTIONS': 10
}

>>> MAX_CONNECTIONS
10

After you call load_config() with the config.json file as an argument, you get the configuration parameters loaded as constants into your global scope. Now, you can use these constants directly in your code.

Introspecting Objects

In programming, type introspection is the ability of a program to inspect an object’s type and properties at runtime. Everything in Python is an object, so being able to examine types and properties at runtime is a valuable asset.

Here are a few built-in functions that allow you to perform some kind of type introspection in Python:

Function Description
id() Returns the identity of an object
dir() Returns a list of names in the current local scope or a list of object attributes
vars() Returns the __dict__ attribute for a module, class, or object

In the following sections, you’ll learn how these functions work and how you can use them in code to perform type introspection. To kick things off, you’ll start with the id() function.

Knowing an Object’s Identity: id()

In Python, every individual object has an associated identity. This identity is a unique and constant integer that identifies the object during its lifetime. Two objects with non-overlapping lifetimes may have the same identity.

If you ever need to know the identity of a given object, then you can use the id() function with the object as an argument:

Python
>>> id(42)
4315605776

>>> id("Python")
4315120464

>>> def greet():
...     print("Hello, World!")
...
>>> id(greet)
4307259040

>>> class Demo:
...     pass
...
>>> id(Demo)
4892672720

When you call id() with any Python object as an argument, you get a number that is the object’s identity. In the CPython implementation of Python, an object’s identity is also the memory address where that object lives.

Knowing an object’s identity may be of great help when you’re debugging your code. For example, say that you want to write code that computes individual values from a kind of Fibonacci sequence. You can do this in many ways. However, you think of using a class with callable instances and an instance attribute that allows you to cache already computed values.

Here’s a possible implementation of this class:

Python fibonacci.py
class Fibonaccish:
    def __init__(self, initial_value=1):
        self._cache = [0, initial_value]

    def __call__(self, index):
        if index < len(self._cache):
            fib_number = self._cache[index]
            print(f"{index} {fib_number} id = {id(fib_number)}")
        else:
            fib_number = self(index - 1) + self(index - 2)
            self._cache.append(fib_number)
        return fib_number

In the initializer method of Fibonaccish, you define a list to hold the cache of computed values. Then, you define the .__call__() special method, which allows the instances of your class to be callable like functions.

In this method, you determine whether the Fibonacci value for the target index is already computed and stored in the cache. Then, you add a call to print() that will help you debug your code using id() to ensure the cached values are used.

Here’s how this class works in practice:

Python
>>> from fibonacci import Fibonaccish

>>> fibonacci_333 = Fibonacci(333)

>>> fibonacci_333(2)
0 0 id = 94800819952840
1 333 id = 140276932935312
333
>>> fibonacci_333(4)
2 333 id = 140276932934960
1 333 id = 140276932935312
2 333 id = 140276932934960
999

In this code example, you create an instance of Fibonaccish with an initial value of 333. The first values of this sequence will be 0, 333, 333, 666, 999, and 1665.

The instance you create is callable, so you can use it as a regular function. Then, you call the instance with 2 as an argument. The call prints the identity of the 0 and 333 values at index 0 and 1, respectively. Next, you call fibonacci_333() with 4 as an argument. In this case, you get the identity of 333 three times, both at index 1 and 2.

When you look at the identities, you realize that your function uses the same object for the same indices, while it’s different for the two different instances of 333. This way, you can confirm that the function uses the cache as expected.

If you repeat the example with the usual Fibonacci sequence, Fibonacci(1), you’ll see slightly different results. In this case, Python will intern 1 under the hood, so that the same object is used at both index 1 and 2 in your cache.

Checking Names and Attributes: dir() and vars()

Sometimes, you need to know the attributes or methods defined in a given object or scope. For this kind of requirement, Python has two different built-in functions, dir() and vars().

The dir() function with no arguments returns the list of names in the current scope. So, the result will depend on the place in which you call the function. With any Python object as an argument, dir() attempts to return the list of attributes for that object.

For example, if you call dir() with no arguments in a fresh REPL session, then you’ll get an output like the following:

Python
>>> dir()
[
    '__annotations__',
    '__builtins__',
    '__doc__',
    '__loader__',
    '__name__',
    '__package__',
    '__spec__'
]

As you can see, dir() returns the list of names that are defined in your current scope, which is the global scope in this example.

If you call dir() with a Python object as an argument, then you’ll get a list of the object’s attributes and methods. If the input object is a class, then you’ll get a list of methods and class attributes. If the object is an instance of an existing class, then you’ll get a list of methods, class attributes, and instance attributes.

Consider the following example that reuses your Rectangle class from the section on the super() function:

Python
>>> class Rectangle:
...     def __init__(self, length, width):
...         self.length = length
...         self.width = width
...     def area(self):
...         return self.length * self.width
...     def perimeter(self):
...         return 2 * (self.length + self.width)
...

>>> dir(Rectangle)
[
    '__class__',
    '__delattr__',
    '__dict__',
    ...
    'area',
    'perimeter'
]

>>> rectangle = Rectangle(2, 4)
>>> dir(rectangle)
[
    '__class__',
    '__delattr__',
    '__dict__',
    ...
    'area',
    'length',
    'perimeter',
    'width'
]

In the first call to dir(), you get the class attributes and methods of the Rectangle class. In this case, you use the class object as an argument. In the second call to dir(), you use an instance of Rectangle as an argument and get all the methods, class attributes, and instance attributes.

The default dir() behavior is different for different types of objects. Here’s a summary of these differences:

Object Behavior
A module object Returns the list of names defined in the module.
A type or class object Returns the list of names of class attributes and methods and of the base classes.
Other objects Returns the list of attributes and methods, including the class attributes and the attributes of the base classes.

You can also customize the default behavior of dir() by providing the .__dir__() special method in your custom classes. However, this topic is beyond the scope of this tutorial.

The vars() function returns the .__dict__ attribute for a module, class, instance, or any other object with a .__dict__ attribute:

Python
>>> vars(Rectangle)
mappingproxy(
{
    '__module__': '__main__',
    '__init__': <function Rectangle.__init__ at 0x10352d080>,
    'area': <function Rectangle.area at 0x10352d120>,
    'perimeter': <function Rectangle.perimeter at 0x10352d1c0>,
    '__dict__': <attribute '__dict__' of 'Rectangle' objects>,
    '__weakref__': <attribute '__weakref__' of 'Rectangle' objects>,
    '__doc__': None
})

>>> vars(rectangle)
{'length': 2, 'width': 4}

In this example, you call vars() with the Rectangle class as an argument. You get the .__dict__ attribute of the class, which contains methods and class attributes. Note that it also contains a .__dict__ attribute that holds the attributes of the class’s instances. That .__dict__ is what you get when you call vars() with an instance of the class.

The .__dict__ attribute is a dictionary that works as a namespace that maps names to objects. For example, it can map a method name to a method object or an attribute name to a specific value or object.

Running Python Code From Strings

In rare situations, it may be useful to evaluate expressions or run code that comes as a string object. This practice isn’t common in real-world code because it may not be secure, especially when the target code comes from an untrusted source, such as the user’s input.

Regardless of the security issues involved, Python has three built-in functions that allow you to evaluate expressions or run code that comes as a string. Here’s a summary of these functions:

Function Description
eval() Evaluates arbitrary Python expressions from a string or compiled code input
exec() Executes arbitrary Python code from a string or compiled code input
compile() Generates a compiled code object from a string

In the following sections, you’ll learn the basics of these functions. To kick things off, you’ll start by using the eval() function to evaluate Python expressions.

Executing Expressions From Strings: eval()

In Python, an expression is a combination of objects and operators that returns a value. You’ll find several types of expressions, including math, Boolean, comparison, bitwise expressions, and more. When working with expressions, you can run them as regular Python code. However, what if you need to evaluate expressions defined as strings?

For example, think of how you would evaluate the following:

Python
"sum([2, 3, 4, 5]) / 4 + 100"

If you want to evaluate this string as an expression, then you’ll have to parse the string and figure out how to extract the operands and operators. Then, you can reconstruct the expression and run it in the Python interpreter. This process may sound like a quick thing to do. However, it can be overwhelming in practice, especially if you consider the infinite number of different expressions that you may need to evaluate in real code.

Fortunately, Python has a built-in eval() function that helps you evaluate expressions that come as strings.

If you have a string that holds a valid Python expression, then you can call eval() with that string as an argument. The function will parse the string, compile it into bytecode, and finally evaluate it as a normal expression:

Python
>>> eval("sum([2, 3, 4, 5]) / 4 + 100")
103.5

Wow! That was quick and smooth! You just passed your string to eval(), ran the code, and got the expression’s result.

The signature of eval() looks something like the following:

Python
eval(expression[, globals[, locals]])

The first argument, expression, holds the expression that you need to evaluate. The rest of the arguments are optional. That’s why they’re enclosed in square brackets. Here’s a summary of these arguments and their meaning:

Argument Description
expression A string holding a valid Python expression
globals A dictionary holding a global namespace to use in the call to eval()
locals A dictionary holding a local namespace to use in the call to eval()

You’ve already seen an example of using the expression argument, so now you can focus on the other two arguments. In each example, you’ll need to provide an expression.

Here’s an example of using the globals argument:

Python
>>> numbers = [2, 3, 4, 5]
>>> n = len(numbers)

>>> eval("sum(numbers) / n + 100")
103.5

>>> eval("sum(numbers) / n + 100", {})
Traceback (most recent call last):
    ...
NameError: name 'numbers' is not defined

>>> eval("sum(numbers) / n + 100", {"numbers": numbers, "n": n})
103.5

By default, eval() has access to the global scope, so you can use all the names defined in this scope in the expression you pass to the function. If you set globals to an empty dictionary, then you restrict access to the global scope, and the function fails.

Finally, you can use an explicit dictionary, as you did in the final example, to provide the global variables that you want to use when evaluating the target expression.

The locals argument works similarly. It takes a dictionary of local names:

Python
>>> def evaluator(expression):
...     numbers = [2, 3, 4, 5]
...     n = len(numbers)
...     return eval(expression, {}, {"numbers": numbers, "n": n})
...

>>> evaluator("sum(numbers) / n + 100")
103.5

Inside the evaluator() function, you define numbers and n as local variables. In the call to eval(), you use an empty dictionary for globals, and a dictionary containing the local variables for locals.

Even though the eval() function may seem like an amazing tool, you must be careful when using it in your code. In practice, you’ll be safer if you don’t use this tool in real-world code. Why?

The eval() function has security implications that can be hard to circumvent. For example, if you use the function to evaluate expressions provided by external users, then you expose your system to the execution of arbitrary Python code.

For more insights on how to reduce the security risks associated with eval(), check out the Minimizing the Security Issues of eval() in the Python eval(): Evaluate Expressions Dynamically tutorial.

Running Code From Strings: exec() and compile()

The eval() function is a powerful tool in Python. However, it’s designed to evaluate expressions. Sometimes, you may want to run more complex pieces of code that come as strings. For example, you may want to run loops, conditional statements, compound statements, and even entire scripts. In this scenario, you can use the built-in exec() and compile() functions.

Here’s the signature of the exec() function:

Python
exec(code [, globals [, locals]])

The code argument can be a string containing valid Python code. It can also be a compiled code object, which you can create with the compile() function. You’ll learn about compile() in a moment. For now, you’ll use a string to provide the code argument.

If code comes as a string, then exec() parses it as a sequence of Python statements. Then, it compiles the code into bytecode, and finally, it executes the code unless a syntax error occurs during the parsing or compilation step.

Consider the following example:

Python calculations.py
>>> functions = [
...     "def add(a, b): return a + b",
...     "def subtract(a, b): return a - b",
...     "def multiply(a, b): return a * b",
...     "def divide(a, b): return a / b",
... ]

>>> for function in functions:
...     exec(function)
...

In this example, you define a list of strings. Each string holds Python functions for a basic arithmetic operation. Then, you start a loop over the list. With exec(), you execute the strings that define the functions. This step brings every function to your current global scope. Now, you can use them as you would a regular function:

Python
>>> add(1, 2)
3
>>> subtract(3, 2)
1
>>> multiply(2, 3)
6
>>> divide(6, 3)
2.0

The arithmetic functions are now available in your global scope, so you can use them to run your calculations.

Like eval(), exec() takes the globals and locals arguments, which again are optional. These arguments have similar meanings in both functions, so you can give them a try as an exercise.

When you have a string containing code that you’ll reuse several times, you can use the compile() function to compile the code once and use it everywhere. This practice will make your code more efficient and fast because the compilation step runs only once.

The signature of compile() looks something like the following:

Python
compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)

This signature is a bit involved because it has several arguments that you need to understand. Here’s a summary of the arguments and their meaning:

Argument Description
source Holds the code that you need to compile into bytecode
filename Hold the file from which the code was read
mode Specifies what kind of code must be compiled
flags and dont_inherit Controls which compiler options should be activated and which future features should be allowed
optimize Specifies the compilation’s optimization level

To read from a string object, you’ll have to set filename to the "<string>" value. The mode argument can take one of the following values:

  • "eval" when source consists of a single expression
  • "exec" when source is a sequence of statements
  • "single" when source is a single interactive statement

Depending on the source code and the function you plan to use to run it, you’ll select the first or the second value. The single argument comes in handy when you want to run a statement like print("Hello, World!") that you’d typically run in an interactive session.

To illustrate how to use compile(), consider the following toy example:

Python
>>> code = """
... result = sum(number for number in iterable if not number % 2)
... """

>>> compiled_code = compile(code, "<string>", "exec")

>>> context = {"iterable": [1, 2, 3, 4]}
>>> exec(compiled_code, context)
>>> context["result"]
6

>>> context = {"iterable": [10, 40, 50, 20]}
>>> exec(compiled_code, context)
>>> context["result"]
120

In this example, you have a piece of code in a string. The code consists of a call to sum() that wraps a generator expression that takes an iterable of numbers and returns the even numbers. Next, you use the compile() function to compile the string into a code object ready for execution. The context dictionary holds an iterable of numbers.

You call exec() with the compiled code and the context dictionary as arguments. Note that you use context to provide the globals argument. The call to exec() will update this dictionary with any name that you define in the compiled code. In this specific example, context ends up holding the result variable with the sum of the even numbers in the iterable.

To access the computed value, you use the context dictionary with the "result" key. In the final example, you reuse the compiled code to perform a similar computation with a different list of values.

Using Miscellaneous Functions

Python has a few other built-in functions that cover miscellaneous topics. Here’s a summary of these functions:

Function Description
help() Invokes the built-in help system
hash() Calculates the hash value of an object
__import__() Invoked by the import statement
memoryview() Returns a memory view object

In the following sections, you’ll learn the basics of these functions and how to use them in your Python code or in an interactive session of the language. To kick things off, you’ll start with the built-in help() function.

Accessing the Built-in Help System: help()

The built-in help() function comes in handy when you’re working in a Python REPL session. This function gives you access to the built-in interactive help system. Go ahead and open an interactive Python session in your terminal. Then, call help() with no arguments. You’ll be presented with the help system:

Python
>>> help()

Welcome to Python 3.x's help utility!

If this is your first time using Python, you should definitely check out
the tutorial on the internet at https://docs.python.org/3.x/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics".  Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".

help>

The output gives you a warm welcome to the Python help utility. Then, it suggests that you take the official Python tutorial if you’re new to the language. In the third and fourth paragraphs, you’re given instructions for using the help system.

At the end of the page, you have the help> prompt waiting for your input. Go ahead and type in the str name and then press Enter. You’ll see something like the following:

Text
Help on class str in module builtins:

class str(object)
 |  str(object='') -> str
 |  str(bytes_or_buffer[, encoding[, errors]]) -> str
 |
 |  Create a new string object from the given object. If encoding or
 |  errors is specified, then the object must expose a data buffer
 |  that will be decoded using the given encoding and error handler.
 |  Otherwise, returns the result of object.__str__() (if defined)
 |  or repr(object).
 |  encoding defaults to sys.getdefaultencoding().
 |  errors defaults to 'strict'.
 |
 |  Methods defined here:
 |
 |  __add__(self, value, /)
 |      Return self+value.
 |
 |  __contains__(self, key, /)
 |      Return key in self.
 ...

This is the help page for the str class. On this page, you’ll find detailed information about the class and its goals. To leave the page and get back to the help system, go ahead and press Q.

While in the help system, you can consult the help for many Python objects, including built-in modules, functions, keywords, and much more. Go ahead and give it a try. You may find some useful information!

There’s a second way you can use the help() function. First, type Q at help> prompt and then press Enter to get back to your interactive session. Once in there, you can call help() with any Python object as an argument. For example, if you call the function with the str class as an argument, then it’ll take you to same page you saw before:

Python
>>> help(str)

This way of calling help() gives you quick access to the help page of a given object. It’s important to note that you can either use the object directly as an argument to help() or the name of the object as a string like in help("str"). However, in most cases, using the name of the object as a string is safer:

Python
>>> help(sys)
Traceback (most recent call last):
    ...
NameError: name 'sys' is not defined

In this example, you try to access the help page for the sys module. You use the module’s name as an argument and get a NameError exception because the module isn’t present in your current scope. The call to help() will work safely if you use the module’s name as a string, as in "sys". Go ahead and give it a try!

Creating Hash Codes: hash()

If you work in fields like data integrity, security, or cryptography, then you may be familiar with hash codes. A hash code is a number that can act as a digital fingerprint for a given piece of data. It’s usually much smaller than the original data and lets you verify its integrity.

To create a hash code for a given object, you need a hash function. Python has its own built-in function for creating hash codes. The function is conveniently called hash().

The signature of hash() is like the following:

Python
hash(object)

It takes an object as an argument and returns the hash value of the input object. The hash value has to be an integer number.

Here are a few examples of using the hash() function:

Python
>>> hash(42)
42
>>> hash(2.7)
1614090106449586178

>>> hash("Hello")
-6239611042439236057

>>> hash(int)
270201092

>>> class DemoClass: pass
...
>>> hash(DemoClass)
346520709

>>> demo_instance = DemoClass()
>>> hash(demo_instance)
271491289

In these examples, you’ve used the hash() function with different objects, including numeric values, strings, function objects, and custom classes. In all cases, you get a unique hash code.

In practice, there are objects that don’t have a hash value:

Python
>>> hash([1, 2, 3])
Traceback (most recent call last):
    ...
TypeError: unhashable type: 'list'

>>> hash({"one": 1, "two": 2})
Traceback (most recent call last):
    ...
TypeError: unhashable type: 'dict'

>>> hash({"red", "green", "bleu"})
Traceback (most recent call last):
    ...
TypeError: unhashable type: 'set'

You’ll note that mutable objects aren’t hashable in Python because you can change the value of a mutable object during its lifetime.

Importing Objects From String Names: __import__()

Python’s built-in __import__() function is an advanced tool that isn’t common in everyday programming. The function is called internally by the import statement. The direct use of __import__() is discouraged in favor of importlib.import_module(). However, it’s a built-in function, so you’ll learn a bit about it in this section.

The signature of __import__() looks something like the following:

Python
__import__(name, globals=None, locals=None, fromlist=(), level=0)

With this function, you can import a module by its name. This name should be a string. Here’s a summary of the function’s arguments and their meaning:

Argument Description
name A module’s name as a string
globals A dictionary representing the global namespace
locals A dictionary representing the local namespace
fromlist A list of objects or submodules that should be imported from the module
level A positive value indicating the number of parent directories to search relative to the directory of the module calling __import__()

To do something equivalent to import sys with the __import__() function, you can do something like the following:

Python
>>> sys = __import__("sys")
>>> sys
<module 'sys' (built-in)>

In this example, you create a sys variable and assign it the result of calling __import__() with the string "sys" as an argument. This call to __import__() imports the sys module into your current global scope.

Manipulating Binary Data Efficiently: memoryview()

The built-in memoryview() function allows you to access the internal data of an object that supports the buffer protocol, such as array.array, bytes, and bytearray objects. You can use this function for manipulating large datasets or interfacing with binary data.

For example, if you have a large dataset and want to use a chunk of it, then creating a copy would be inefficient. Instead, you can make a memoryview object to access the data without copying it. This allows you to use less memory and increases the execution speed.

For example, say that you have the pixel data of an image represented as a bytearray, and you want to invert the pixel values. To do this operation efficiently, you can use the memoryview() function:

Python
>>> image = bytearray([0, 127, 255, 64, 128, 192, 32, 96, 160])
>>> mv = memoryview(image)

>>> for i in range(len(mv)):
...     mv[i] = 255 - mv[i]
...

>>> list(mv)
[255, 128, 0, 191, 127, 63, 223, 159, 95]
>>> list(image)
[255, 128, 0, 191, 127, 63, 223, 159, 95]

In this example, you create a memoryview object to access the data that represents your image. In the for loop, you iterate over the data and invert the pixel values. The transformation reflects on the original data.

In short, the memoryview() function is a powerful tool for working with objects that support the buffer protocol without copying the data, which makes your code memory-efficient and faster.

Conclusion

You’ve learned the basics of Python’s built-in functions. These are functions that you can use directly without importing anything because they’re available in the built-in scope or namespace.

Built-in functions solve a wide range of common programming problems such as performing math operations, working with common data types, processing iterables of data, handling input and output, working with scopes, and more.

In this tutorial, you’ve learned:

  • The basics of Python’s built-in functions
  • About common use cases of Python’s built-in functions
  • How to use these functions to solve practical problems in Python

With this knowledge, you’ve got foundational Python skills that will help you write robust Pythonic code. More importantly, you’re now aware of all these incredible functions and can use them in your code to tackle common tasks efficiently without reinventing the wheel.

Take the Quiz: Test your knowledge with our interactive “Python's Built-in Functions: A Complete Exploration” quiz. You’ll receive a score upon completion to help you track your learning progress:


Interactive Quiz

Python's Built-in Functions: A Complete Exploration

Take this quiz to test your knowledge about the available built-in functions in Python. By taking this quiz, you'll deepen your understanding of how to use these functions and the common programming problems they cover, from mathematical computations to Python-specific features.

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Leodanis Pozo Ramos

Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites.

» More about Leodanis

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!