Introduction to Context Managers
00:00 In the previous lesson, I gave an overview of the course. In this lesson, I’ll be showing you the how and why of context managers.
00:08 This is the problem. Consider the three lines of code that you need to open and write to a file. If this line fails, resulting in an exception, then this line won’t run.
00:20 If the exception results in your script exiting, the operating system will typically close the file for you. But what if you catch the error somewhere else to inform the user? Well, you’ve still got an open file handle.
00:32 Your operating system has a maximum number of file handles that it will allow you to open at any given time. It typically isn’t a small number. On many Linux boxes, it defaults to just over a thousand, but if you’re executing a long-running program, lack of resource management can catch up with you.
00:50
Dad joke in the title aside, a common way of dealing with the problem is to use a try … except … finally block. The finally portion of the block runs whether or not there was an exception, guaranteeing that your file gets closed.
01:04 The one drawback of this approach is you have to remember to do it each and every time you deal with a resource. Spoiler alert, context managers don’t have this challenge.
01:16
The Python open() function can be used as both a function and as a context manager. To use it as a context manager, you write the with keyword.
01:25
When you instantiate open() as a context manager, it returns a file handle. That’s seen here, after the as keyword. with statements form a code block.
01:36
Everything indented underneath it is in the block. When the block is finished, the context manager automatically closes the file for you. This is in contrast to the try … except … finally case. Here, you don’t have to remember to handle the resource.
01:51
It’s done by the context manager. Not having to remember is good. The downside is you don’t have an opportunity to deal with the exception. It gets caught, and the file gets closed, but you can’t inform the user like you would with the except portion of the try block.
02:08
Context managers are built following a protocol. They are objects, like everything in Python, and these objects need to have a .__enter__() method, which, if it returns something, becomes the target of the as portion of the statement, and a .__exit__() method that is called when the context block goes out of scope.
02:27
This is where your resource closing goes. When Python hits a with statement, it calls the context manager’s .__enter__() method, provides the result of .__enter__() to the as target, runs the code block underneath the with, and finally, once the block exits, it calls the context manager’s .__exit__() method.
02:49 This method actually takes arguments that give information about whether there was an exception, but I’ll get back to that later.
02:57
Python 3.1—yep, that’s just a single 1; this is quite some time ago—introduced the capability of instantiating multiple context managers in the same with block.
03:08 This example opens a file for reading and opens a different file for writing. This is a common pattern when you’re doing data transformation. A little more on this in a future lesson.
03:21 Next up, I’ll give you a tour of some context managers from the standard library and how they’re used.
Christopher Trudeau RP Team on May 5, 2024
Hi dosinboyejo,
Generally speaking, once the program exits, the OS will clean up for you. This isn’t always guaranteed – your program crashing can cause some weirdness. Best practice is to handle the situation properly.
I believe what I was attempting to say here, was if you are catching it, and your program hasn’t crashed, then the file handle will continue to stick around. For a single file handle that isn’t a problem, but for programs that are running for a long time and using a lot of files (remember things like network connections are also file handles), you’ll run into a resource limitation.
Best practice is to make sure your file handles are closed as soon as they can be and to ensure they get closed even if an exception gets raised.
Become a Member to join the conversation.
Doyin on May 5, 2024
This bit is not clear:
00:20 If the exception results in your script exiting, the operating system will typically close the file for you. But what if you catch the error somewhere else to inform the user? Well, you’ve still got an open file handle.
Christopher says that the OS typically closes the file in the event of an error. He then goes on to describe a scenario where the error occurs further along in the program, saying you’ve still got a file open.
Why would the file be closed in one event of an error and remain open in the other?