More Powerful F-Strings

00:00 In the previous lesson, I gave a quick overview of the course. In this lesson, I’m going to talk about improvements made in Python 3.12 to f-strings. I guess I’m just getting old. It seems like not that long ago, I adopted f-strings in my code, but it turns out they’ve been around since Python 3.6.

00:19 If you haven’t used them before, they’re an improvement over C-style parametrized strings as well as the .format() function, which somehow I just never got around to adopting.

00:29 When f-strings were added to Python, the base code of the existing parser wasn’t capable of parsing them, so a separate parser was added explicitly for their case.

00:40 As a result, they technically weren’t part of the official grammar. Python 3.9 converted to a more powerful parser called the PEG. It’s the magic behind a lot of the goodness you’ve seen in the last couple of versions, especially with error message improvements.

00:56 The newer parser is perfectly capable of parsing f-strings, so in Python 3.12, the f-string parsing was reimplemented to use the PEG. For the core maintainers, this is a big advantage.

01:08 It means the old secondary parsing code can be removed. A byproduct of the new parser is some former restrictions on f-strings have been lifted. Let’s head off to the REPL, and I’ll show you the new changes.

01:24 To demonstrate the new stuff, I’m going to compare and contrast 3.11 with 3.12. The top window will have 3.11, while the bottom window will have 3.12. The first change I’m going to talk about is how quotes work inside of an f-string. Let me just create a dict here,

01:48 and in order to reference a value from the dict inside an f-string, I need to use a nested quote. In 3.11, the outer and inner quotes have to be different, following the same rules as other kinds of strings.

02:05 If I try to use the same kind of quote inside, it falls down. This restriction was there because the old f-strings were identified as strings by the lexer, then passed to the custom parser for further processing.

02:24 They had to be valid strings, or the lexer would complain. This restriction is gone in 3.12. Same dict.

02:41 The old way still works, and the new way is also allowed. Frankly, I’m not sure if I like this. I get that it can be helpful with very complicated f-strings, but I’m so accustomed to mentally scanning a string that this looks weird to me.

03:00 Maybe I’ll get used to it. It also means the IDEs and editors out there are going to need to have their tokenization updated. Otherwise, the syntax highlighting is going to be wrong in this case.

03:13 Several of my colleagues are already struggling with linters that are flagging these kinds of problems. The next restriction lifted has to do with how backslashes work.

03:29 Here, I’ve made a list of words by splitting a string. I can join them with a newline, but I can’t do that inside of an f-string. Backslash characters were not allowed inside an f-string expression.

03:51 To accomplish what I’m attempting here, you need to do it as two different steps.

04:08 Now, on the 3.12 side, this restriction has been lifted. Same words … and the join …

04:25 and now I can join inside the f-string. Again, not sure this is something I’m ever going to do. In fact, I wasn’t aware this was even a restriction. I’d never tried it before.

04:35 But it does mean the expression portion of an f-string is now more consistent with expressions in the language. F-strings can be multiline using triple quotes.

04:53 But the expression portion isn’t allowed to have comments.

05:07 This is now possible in 3.12.

05:22 This one I like. The introduction of the PEG parser has enabled a vast improvement in syntax errors in Python. I’d say it was actually my favorite feature of both Python 3.10 and 3.11, definitely the one I use the most.

05:36 More improvements have been done in 3.12, which I’ll cover later. But while I’m here, let me show you an f-string-specific one. That error is a bit vague. It tells you there’s an error, but not really what it is. Now in 3.12 land, you get the same kind of error as you’d get on a problematic expression outside of an f-string.

06:04 This is possible because the same parser is now being used in both places. The last thing to show you is a new ability to nest f-strings.

06:33 Not sure I can come up with a use case for this, but well, if you want to, you can do it now. I mentioned that error messages have been improving since the introduction of the PEG parser. That trend continues in Python 3.12.

06:49 I’ll show you some more improvements in the next lesson.