TOML Support

00:00 In the previous lesson, I showed you how traceback information now has detailed indicators that can help you find the cause of your errors. In this lesson, I’ll be showing you TOML, the configuration language that is now part of the standard library.

00:15 TOML is short for Tom’s Obvious, Minimal Language and is a configuration file format. If you’ve ever seen Windows INI files, it’s quite similar to those. TOML’s been around for a long time, but up until 3.11 could only be used in Python with a third-party package.

00:33 It’s now part of the standard library. TOML has become quite prevalent in the Python ecosystem. Many packaging tools use it. You may have seen a pyproject.toml file kicking around, for example.

00:46 With this prevalence, it kind of makes sense to add TOML to the list of included batteries. This is a simple TOML file. Straightforward, huh? With the exception of the fact that true isn’t capitalized, this could be Python. For a lot of configuration data, this might be all you needed. TOML also supports grouping of keys, though.

01:12 This is a more complicated TOML file. You can define a section using square brackets ([]). You can think of this as a grouping of configuration items or, from a Python perspective, nested dictionaries. Note that it also supports lists.

01:27 The peps here contains a series of integers. Let’s go to the REPL and parse this file.

01:39 First off, you need to import the module. The core developers decided to go with the name tomllib to avoid name conflicts with existing TOML libraries.

01:55 To read a file, first I have to open it. One little tricky thing here. TOML files are to be opened in binary mode, which is why the "rb" is being used.

02:06 It’s done this way, as Python wants to enforce that the file is in UTF-8 format. But don’t worry, ASCII is a pure subset, so you don’t have to do anything fancy to create your file.

02:17 Once I’ve got a file handle,

02:23 I parse the contents of the file, calling the .load() method on tomllib. And that’s pretty much it. The result is returned as a Python dictionary.

02:35 Let’s pretty-print this so you can see it a bit better.

02:44 Each section in the TOML file becomes a nested dictionary in the result. Since the data types in TOML are very similar to those in Python, there isn’t much in the way of surprises here. It is even datetime aware, converting the date string in the TOML file into datetime object in Python.

03:02 There’s more than one way to skin a cat. Who came up with that cliche? Probably not a very nice person. pathlib is one of my favorite modules.

03:13 If you’re still going old-school and using os.path, I encourage you to check this one out instead.

03:24 Here, I’ve constructed a new Path object pointing at my TOML file …

03:34 which then I read in using pathlib’s .read_text() method. In addition to the .load() method, TOML also provides a .loads(), short for load string, which will parse the text.

03:51 The result is the same, so you can decide to choose either way of getting your TOML into your Python.

03:59 TOML supports many different data types, including comments, key-value pairs, arrays, tables, inline tables, integers, floats, Booleans, and dates and times.

04:14 TOML becoming part of Python’s standard library is a good thing. It’s being used more and more in the ecosystem. black, mypy, pytest, and many more already use TOML for their configuration. To be consistent with Python 3, only UTF-8-based files are supported, and the interface has been written to be similar to how JSON and pickle work.

04:36 Why am I bringing that random fact up? Well, if you’re using a third-party TOML library, they don’t use that style of interface. The two more popular libraries allow you to load using just the filename.

04:47 Python has decided consistency is better, so you’re stuck using the file handle or .loads() .readtext() mechanism that I showed you. As TOML is primarily used as a configuration format, it has also been decided to not support the writing of files.

05:03 If you want to programmatically generate TOML files, you’re encouraged to use third-party libraries. TOML Kit is one of the more popular one of those. It has the benefit of also being style preserving, which means when you read the file in and then write it out, it will look the same. That sounds like a simple thing, but parsers for a lot of text formats don’t do this.

05:27 Next up, I’ll be showing you how exceptions have improved, along with a new way of using tasks in asyncio. I know that grouping seems random, but they’re related. I promise.