In this lesson, you’ll learn about more precise types in Python 3.8. Python’s typing system is quite mature at this point. However, in Python 3.8, some new features have been added to typing to allow more precise typing:
- Literal types
- Typed dictionaries
- Final objects
- Protocols
Python supports optional type hints, typically as annotations on your code:
>>> def double(number: float) -> float:
... return 2 * number
In this example, you say that number
should be a float and double()
should return a float as well. However, Python treats these annotations as hints. They are not enforced at runtime:
>>> double(3.14)
6.28
>>> double("I'm not a float")
"I'm not a floatI'm not a float"
double()
happily accepts "I'm not a float"
as an argument, even though that’s not a float.
Type hints allow static type checkers to do type checking of your Python code, without actually running your scripts. This is reminiscent of compilers catching type errors in other languages like Java, and Rust. Additionally, type hints act as documentation of your code, making it easier to read, as well as improving auto-complete in your IDE.
Note: There are several static type checkers available, including Pyright, Pytype, and Pyre. In this course, you’ll use Mypy. You can install Mypy from PyPI using pip:
$ python -m pip install mypy
Try out Mypy on the code example from before. Create a new file named float_check.py
:
# float_check.py
def double(number: float) -> float:
return 2 * number
double(3.14)
double("I'm not a float")
Now run Mypy on this code:
$ mypy float_check.py
float_check.py:8: error: Argument 1 to "double" has incompatible
type "str"; expected "float"
Found 1 error in 1 file (checked 1 source file)
Based on the type hints, Mypy is able to tell you that you are using the wrong type on line 8. Change the argument for the second call to double()
to a float
:
# float_check.py
def double(number: float) -> float:
return 2 * number
double(3.14)
double(2.4)
Now run Mypy on this code again:
$ mypy float_check.py
Success: no issues found in 1 source file
In some sense, Mypy is the reference implementation of a type checker for Python, and is being developed at Dropbox under the lead of Jukka Lehtasalo. Python’s creator, Guido van Rossum, is part of the Mypy team.
You can find more information about type hints in Python in the original PEP 484, as well as in Python Type Checking (Guide), and the video course Python Type Checking.
There are four new PEPs about type checking that have been accepted and included in Python 3.8. You’ll see short examples from each of these.
PEP 586 introduced the Literal
type. Literal
is a bit special in that it represents one or several specific values. One use case of Literal
is to be able to precisely add types, when string arguments are used to describe specific behavior. Consider the following example:
# draw_line.py
def draw_line(direction: str) -> None:
if direction == "horizontal":
... # Draw horizontal line
elif direction == "vertical":
... # Draw vertical line
else:
raise ValueError(f"invalid direction {direction!r}")
draw_line("up")
The program will pass the static type checker, even though "up"
is an invalid direction. The type checker only checks that "up"
is a string. In this case, it would be more precise to say that direction must be either the literal string "horizontal"
or the literal string "vertical"
. Using Literal
, you can do exactly that:
# draw_line.py
from typing import Literal
def draw_line(direction: Literal["horizontal", "vertical"]) -> None:
if direction == "horizontal":
... # Draw horizontal line
elif direction == "vertical":
... # Draw vertical line
else:
raise ValueError(f"invalid direction {direction!r}")
draw_line("up")
By exposing the allowed values of direction to the type checker, you can now be warned about the error:
$ mypy draw_line.py
draw_line.py:15: error:
Argument 1 to "draw_line" has incompatible type "Literal['up']";
expected "Union[Literal['horizontal'], Literal['vertical']]"
Found 1 error in 1 file (checked 1 source file)
The basic syntax is Literal[<literal>]
. For instance, Literal[38]
represents the literal value 38
. You can express one of several literal values using Union
:
Union[Literal["horizontal"], Literal["vertical"]]
Since this is a fairly common use case, you can (and probably should) use the simpler notation Literal["horizontal", "vertical"]
instead. You already used the latter when adding types to draw_line()
.
If you look carefully at the output from Mypy above, you can see that it translated the simpler notation to the Union notation internally.
There are cases where the type of the return value of a function depends on the input arguments. One example is open()
, which may return a text string or a byte array depending on the value of mode
. This can be handled through overloading.
The following example shows the skeleton of a calculator that can return the answer either as regular numbers (38
) or as roman numerals (XXXVIII
):
# calculator.py
from typing import Union
ARABIC_TO_ROMAN = [(1000, "M"), (900, "CM"), (500, "D"), (400, "CD"),
(100, "C"), (90, "XC"), (50, "L"), (40, "XL"),
(10, "X"), (9, "IX"), (5, "V"), (4, "IV"), (1, "I")]
def _convert_to_roman_numeral(number: int) -> str:
"""Convert number to a roman numeral string"""
result = list()
for arabic, roman in ARABIC_TO_ROMAN:
count, number = divmod(number, arabic)
result.append(roman * count)
return "".join(result)
def add(num_1: int, num_2: int, to_roman: bool = True) -> Union[str, int]:
"""Add two numbers"""
result = num_1 + num_2
if to_roman:
return _convert_to_roman_numeral(result)
else:
return result
The code has the correct type hints: the result of add()
will be either str
or int
. However, often this code will be called with a literal True
or False
as the value of to_roman
, in which case you would like the type checker to infer exactly whether str
or int
is returned. This can be done using Literal
together with @overload
:
# calculator.py
from typing import Literal, overload, Union
ARABIC_TO_ROMAN = [(1000, "M"), (900, "CM"), (500, "D"), (400, "CD"),
(100, "C"), (90, "XC"), (50, "L"), (40, "XL"),
(10, "X"), (9, "IX"), (5, "V"), (4, "IV"), (1, "I")]
def _convert_to_roman_numeral(number: int) -> str:
"""Convert number to a roman numeral string"""
result = list()
for arabic, roman in ARABIC_TO_ROMAN:
count, number = divmod(number, arabic)
result.append(roman * count)
return "".join(result)
@overload
def add(num_1: int, num_2: int, to_roman: Literal[True]) -> str: ...
@overload
def add(num_1: int, num_2: int, to_roman: Literal[False]) -> int: ...
def add(num_1: int, num_2: int, to_roman: bool = True) -> Union[str, int]:
"""Add two numbers"""
result = num_1 + num_2
if to_roman:
return _convert_to_roman_numeral(result)
else:
return result
The added @overload
signatures will help your type checker infer str
or int
depending on the literal values of to_roman
. Note that the ellipses (...
) are a literal part of the code. They stand in for the function body in the overloaded signatures.
As a complement to Literal
, PEP 591 introduces Final
. This qualifier specifies that a variable or attribute should not be reassigned, redefined, or overridden. The following is a typing error:
# final_id.py
from typing import Final
ID: Final = 1
...
ID += 1
Mypy will highlight the line ID += 1
, and note that you Cannot
assign to final name "ID"
. This gives you a way to ensure that constants in your code never change their value.
Additionally, there is also a @final
decorator that can be applied to classes and methods. Classes decorated with @final
can’t be subclassed, while @final
methods can’t be overridden by subclasses:
# final_class.py
from typing import final
@final
class Base:
...
class Sub(Base):
...
Mypy will flag this example with the error message Cannot inherit from final class "Base"
. To learn more about Final
and @final
, see PEP 591.
The third PEP allowing for more specific type hints is PEP 589, which introduces TypedDict
. This can be used to specify types for keys and values in a dictionary using a notation that is similar to the typed NamedTuple
.
Traditionally, dictionaries have been annotated using Dict
. The issue is that this only allowed one type for the keys and one type for the values, often leading to annotations like Dict[str, Any]
. As an example, consider a dictionary that registers information about Python versions:
py38 = {"version": "3.8", "release_year": 2019}
The value corresponding to version
is a string, while release_year
is an integer. This can’t be precisely represented using Dict
. With the new TypedDict
, you can do the following:
# typed_dict.py
from typing import TypedDict
class PythonVersion(TypedDict):
version: str
release_year: int
py38 = PythonVersion(version="3.8", release_year=2019)
The type checker will then be able to infer that py38["version"]
has type str
, while py38["release_year"]
is an int
. At runtime, a TypedDict
is a regular dict
, and type hints are ignored as usual:
>>> from typed_dict import *
>>> py38
{'version': '3.8', 'release_year': 2019}
>>> type(py38)
<class 'dict'>
Mypy will let you know if any of your values has the wrong type, or if you use a key that has not been declared. See PEP 589 for more examples.
Mypy has supported Protocols for a while already. However, the official acceptance only happened in May 2019.
Protocols are a way of formalizing Python’s support for duck typing:
When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck. (Source)
Duck typing allows you to, for example, read .name
on any object that has a .name
attribute, without really caring about the type of the object. It may seem counter-intuitive for the typing system to support this. Through structural subtyping, it’s still possible to make sense of duck typing.
You can for instance define a protocol called Named
that can identify all objects with a .name
attribute:
# protocol.py
from typing import Protocol
class Named(Protocol):
name: str
def greet(obj: Named) -> None:
print(f"Hi {obj.name}")
class Dog:
...
x = Dog()
greet(x)
After creating the Named
protocol, and defining the function greet()
that takes an argument of obj
, the type hint is specifying that obj
follows the Named
protocol. Run the code through Mypy to see what it finds:
$ mypy protocol.py
protocol.py:16: error: Argument 1 to "greet" has incompatible type "Dog"; expected "Named"
Found 1 error in 1 file (checked 1 source file)
The Dog
class did not have an attribute of .name
, and therefore did not meet the check for the Named
protocol. Add a .name
attribute to the class, with a default string:
# protocol.py
from typing import Protocol
class Named(Protocol):
name: str
def greet(obj: Named) -> None:
print(f"Hi {obj.name}")
class Dog:
name = 'Good Dog'
x = Dog()
greet(x)
Run protocol.py
through Mypy again:
$ mypy protocol.py
Success: no issues found in 1 source file
As you have confirmed, greet()
takes any object, as long as it defines a .name
attribute. See PEP 544 and the Mypy documentation for more information about protocols.
Shehu Bello on March 26, 2020
Very Interesting features. However, with these features, python is looking like a statically typed languages such as C.