2.6 → 3.0 - Transitioning to Python 3

00:00 In the previous lesson, we covered the with keyword and defaultdict from collections. In this lesson, we’ll talk about the 2 to 3 transition and what it meant to go from Python 2.7 to 3.0.

00:13 So Python 2.6, which is from 2008 to 2013, introduced the format() method for strings allowing you to kind of go a little bit further than the C-style formatting, adding the curly brackets and then also exceptions could be caught, if you will, with an except with as statement.

00:34 And also were added class decorators building on top of what we already were able to use with function decorators.

00:42 To learn a little more about raising and handling Python exceptions, we have not only an excellent tutorial but a video course as well.

00:53 Python 2.7 had quite the long life. This is about the time that I started looking at Python and I actually asked some employers, what version of Python are you guys using?

01:04 It was always kind of a common question around this era and almost everybody I was speaking to was already working on Python 3. And so that was nice to not have to learn about the differences in how to kind of go into it.

01:16 So this is kind of a, a bit of a historical review, which it may be for many of you watching this video course right now. The release actually overlapped 3.0 all the way up to 3.8 and it added some additional things.

01:30 Not only did we now have the list comprehensions spoken about earlier, but we had dictionary and set comprehensions also in Python 2.7. argparse was added to help you work through your command-line arguments there, which in some ways was intended to replace optparse and some of 3.0 and 3.1 features also got backported so that they would function inside of Python 2.7.

01:55 Yeah, I almost wish that I had started a little later because just, 2.7 was the only choice when I started. I spent a lot of time and energy shifting to 3.

02:06 If I’d put off my Python journey by a year or two, I might have made my life easier and now like us young people here, just kidding.

02:15 Which leads us to 3. With three came a lot of changes. Python 3 was a big shift. Up until now, each of the versions had been backwardly compatible. You could run your 2.2 script in a 2.7 interpreter without any problem, but Python 3 decided to change that.

02:31 A big justification for that was the move to Unicode. The original Python was ASCII only. Python 2.0 added the u prefix so that you could write Unicode strings, but the underlying code itself still had to be ASCII.

02:45 Making Python Unicode by default was a big modernization step, but it made compatibility a challenge and since they were breaking backward compatibility, they decided to fix a few other things as well.

02:56 Because, well, broken is broken, you might as well do a bigger cleanup. One of those changes was to move print from being a statement to a function.

03:04 This made it more in line with other aspects of the language and removed some complexity from the parser. So one of the things I’ve wondered about in this transition from the print statement to the print() function is what was missing and what were the benefits of adding it?

03:20 You know, making it into a function instead of the, the simpler, if you will, print statement I feel like. I don’t know what their driver was.

03:27 I suspect some of it was just, you know, keywords are problematic. The parser has to do special things with them, whereas functions are just functions. So that right there, it might be enough reason to change it.

03:37 Right. This also makes certain things easier. So there was a way with the print statement to by default, when you say print(), it sticks a new line on the end of it.

03:45 Yeah, okay. There was a, there was a way with the print statement to stop it from doing that, but it was like this little hackish thing, whereas the print() function, you can specify what the end of line character is, so it made that clearer as well.

03:57 Fundamentally, I think it’s just, it was a special case and that kind of leads you to the question why. You’ve got all these built-in functions. Why is print different and, and it, it, it was there for historic reasons as far as I can tell.

04:10 Another change was to switch a number of functions and methods that return lists to things that return iterators. This can actually be a big performance gain.

04:19 For example, if I call range from one to a thousand with a list, I get the whole thing even if I don’t use all of it. With an iterator, if I break out of a for loop partway through, the rest of the form doesn’t get generated.

04:32 This is good for memory and for speed. We’ll be touching more on that when we get to the yield and yield from keywords that got added later.

04:40 Part of that cleanup that I was mentioning was also reorganizing the standard library for consistency. Several modules, which had capitalized names, got renamed to match the rest of the library with small letters.

04:53 And several modules got merged together for clarity. For example, urllib2, urlparse and robotparser all got put together under the urllib module. To help with transitioning code, a module named six got created.

05:08 The name comes from two times three, so two to three. There you go. Funny programmers can be funny.

05:15 If you needed your code to run in both Python 2 and 3, you would import six and it would map your import to the right place in the interpreter when you were running.

05:23 So you could say from six import queue with the small Q, and if you were in 2, it would do the capital for you and figure all that out.

05:30 So it was a nice little sort of wedge. I, I remember when I started seeing this in so many libraries, importing six was like such a common thing and it’s starting to disappear.

05:41 Yeah, it was pretty much everywhere. You take something, you know, a long-lived library, like something like Django and they’ve got long-term support standards that are multiple years.

05:50 So you know, there were versions of Django that had to be in both the 2 and the 3 world and you don’t want to maintain two copies of the code. So you start putting these kinds of little shims in, in order to try and deal with it.

05:59 Right? Yeah. And along the same lines, the 2 to 3 script was a semi-automatic way of translating your code. You ran it on your Python 2 files and it output Python 3.

06:10 It wasn’t perfect, but it was better than doing it manually.

06:14 The other change that I want to highlight was the introduction of keyword-only arguments. Before this, any argument to a function could be called by its position or by its keyword.

06:23 Now by putting an asterisk in the declaration, any argument after it could only be called with a keyword. So in this example, I couldn’t just pass in size, I have to say key=size or I get an error.

06:36 Let’s head to the REPL and explore some of the differences between Python’s 2 and 3. In the window on top here, I’ve got a 2.7 interpreter going. I’m going to write a little bit of code and then I’ll do the same in 3 to show you the difference. Python one didn’t have classes, but somewhere along the way they got added.

06:58 Nobody was really happy with them though so they got redesigned in Python 2.2. Mr. Bailey referred to this as the new style classes to distinguish them from what was the old style class, which of course now is both the old style classes, which just is confusing.

07:15 Anyhow, in order to allow these to exist side by side for a while, the new style classes had to inherit from object like you see here on the screen.

07:24 Now I’ll add a method and a string.

07:31 Remember, this is an ASCII string because 2.7, that was the default. By contrast, to get Unicode

07:38 with my little patriotic emoji here, I have to have the u prefix in order to get into Unicode mode.

07:46 And there’s that print statement. No parentheses, one more of them and I’ll try it out.

07:57 And there you go. That’s the content.

08:00 Now let’s look at Python three.

08:04 Inheriting from object all the time was a pain, especially since the old style objects were gone. Python 3 being a backwardly incompatible release was a perfect time to clean this up.

08:15 In Python 3, you’re still actually inheriting from object, but the compiler is doing it for you.

08:22 Now, my method, and a name

08:27 and another flag. Since Python 3 is Unicode by default, no need for the u prefix.

08:36 And there’s the more modern print() as a function.

08:42 There you are Joe, with your flag to go. I blame Canada. Blame Canada. Why not? Python 2.7 with the inheritance from object, because it was a mechanism for allowing backward compatibility.

08:56 This is something which caused me problems several times. Having come from other programming languages, I didn’t always remember that I needed to do this.

09:05 And of course if you left it off, you’d get the old old style classes and they didn’t behave the same way and they would work fine until they didn’t. Because you’d go to do some inheritance or something and it wouldn’t work the way you expected it to.

09:18 So I, I ran into this trouble a few times. So yeah, so it was a, it was a dangerous, dangerous bug because it would work right up until the point it didn’t.

09:27 So yeah, I was very happy to see those gone in 3.

09:31 So those are three key differences from Python 2 and 3. This may not seem like a lot, but if you were managing hundreds of thousands of lines of Python, it could be a lot of work to make sure that nothing broke. And all that work is kind of why the transition didn’t go over very well.

09:47 Breaking backward compatibility caused a lot of work, and coders are busy folks. They want to create new things, not have to do a bunch of stuff just to keep the old things running.

09:56 The 2 to 3 script I mentioned was helpful, but it wasn’t perfect, and this was the pre-VI coding days. People were suspicious that an automated process would be effective.

10:06 Folks agreed that some of the changes were needed to modernize the language, but there was a lot of disagreement on whether the amount of work generated was worth the trouble.

10:15 You’ll still see blog posts on how it should have been done, but those are often by folks who weren’t actually maintaining the interpreter. The real problem came from it being all or nothing.

10:25 You were either Python 3 compatible or you weren’t and there was no half pregnant. One of Python’s superpowers is the ecosystem. There are tons and tons of libraries out there, and any large code base is going to be dependent on dozens of them, and that’s where all or nothing became a problem.

10:41 Just because you wanted to use Python 3 didn’t mean you could as some library or other that you were dependent on may not have made the transition yet. Large libraries like Django were slow to adopt.

10:53 The first Django release that supported Python 3 was in 2013. That’s five years after the Python 3.0 release itself. The transition took a while.

11:04 Although I lived through this transition, to remind myself on just what the pain points were, I did a bit of digging on the web. I came across a rather typical flame war in the comment section on an article about the transition.

11:16 Keep in mind, this article was posted after 2.7 was no longer supported. So that’s 12 years into the transition and you still got this kind of vitriol. I miss buggy whips.

11:28 If you are one of those rare souls who are still maintaining 2.7 code, I do encourage you to read this first tutorial to learn how to migrate. Before exposing you to all this trauma, Mr. Bailey mentioned that 2.7 added argparse.

11:41 To learn more about that, you can see my course or this tutorial.

11:47 The next lesson covers how to yield from generators and what it means to mock your tests.