Improved Traceback Messages

00:00 In the previous lesson, I talked about some of the speed improvements in Python 3.11. In this lesson, I’ll show you how tracebacks have become more descriptive.

00:10 The changes described in this lesson are based on PEP 657, called Include Fine-Grained Error Locations in Tracebacks. Python 3.9 changed the underlying parser in CPython, which has enabled some interesting changes in error messages.

00:26 Some of my favorite improvements in Python 3.10 were in the errors, and Python 3.11 continues on this path. Traceback output from exceptions now have additional information pointing to the source of the problem. Let’s go look at some examples.

00:44 To look at the errors, first I’m going to need some code with problems in it. This script takes some data in dictionary format and populates a Person object with it. It’s a bit of a simplification, but this kind of code is common when, say, deserializing JSON.

01:00 Instead of JSON though, I’m starting with a hard-coded list of dictionaries. Each item in the list is a dictionary itself containing name, date of birth, and date of death data.

01:13 Some of the items are missing data. This is where the errors are going to come from. Let me just scroll down here. The data is to be converted into a Person object.

01:27 The person is based on a NamedTuple and contains a str field for their name and a tuple for their birth and death dates.

01:37 The dict_to_person() utility function constructs a new Person object based on a dictionary passed in, and the convert_pair() function, which I’ll admit is a bit of a contrived example, converts two at a time.

01:51 I foresee twice the failure in my future. Okay, let’s go break some things. To see the improvements, I’m going to run the previous script in both Python 3.10 and 3.11.

02:05 I’ll start with 3.10 in the top window here. I’ve used the -i argument to Python, which loads the script and then kicks you into a REPL. Remember that the script only defined the data and functions.

02:22 It didn’t run anything. I’ll be doing that in the REPL.

02:31 Here, I’ve started by converting my first scientist. So far, so good. Dictionary in, Person out. For the second scientist, I’ll be using 'Euclid', who has some information missing.

02:52 When I try to convert 'Euclid', I get an exception and a traceback printed out as a result. The traceback tells me the failure was a KeyError and happened on line 37 in the dict_to_person() function.

03:04 It also shows me line 37, which is an f-string where the key 'last' was used but unavailable. In the bottom window, I’m going to run the same script, this time with Python 3.11.

03:19 Same -i flag … same happy case … same 'Euclid' …

03:36 and the same error, but this time with more information. The squiggly line made up of the tilde character (~) is new. It shows the dictionary where the KeyError happened, while the line of up arrows, or carets (^), shows the problematic key. Handy, huh?

03:53 Not earth-shattering, but sometimes the simple things can make your life easier. Okay, back to Python 3.10, let’s break something else. New scientist, this one having the right key but a None value.

04:14 And you get a different kind of exception. This time, it’s a TypeError. Let’s see this in Python 3.11. 'Nasr' again … and once again, the error is annotated with the two funky lines showing the dictionary and the specific item in the dictionary that caused the problem. All right, back to 3.10 for another go …

05:02 And here’s a case where these indicators become really helpful. In Python 3.10, there was no way of knowing which of the two references to "year" in line 38 caused the problem. In Python 3.11, This is made clear with the squiggly lines. All right, it’s time to double down.

05:27 Here, I’ve used the convert_pair() function that converts two scientists at a time.

05:39 When I use bad data, it gets pretty hard to know what’s going on. Which call to the function caused the problem? I’m sure you’re way ahead of me at this point, but here’s the Python 3.11 version.

05:59 Not only does the error in line 38 get annotated, but so does the source of the error in line 44. You can now see which of the two calls broke, which gives you insight into which piece of data is problematic.

06:16 Enough errors for now, more to come later. Next up, I’ll show you TOML, the configuration language which is now part of the standard library.