How "with" is better than try/catch to open a file in Python?

Question

I got that the with statement help you to turn this:

try:     f = open(my_file)     do_stuff_that_fails() except:     pass finally:     f.close() 

Into:

with open(my_file) as f:     do_stuff_that_fails() 

But how is that better? You still have to handle the case with the file not being able to be opened (like prompting the user to tell him he doesn't have permissions), so in reality you'd have:

try:     with open(my_file) as f:         do_stuff_that_fails() except (IOError, OSError, Failure) as e:     do_stuff_when_it_doesnt_work() 

Which is equivalent to:

try:     f = open(my_file)     do_stuff_that_fails() except (IOError, OSError, Faillure) as e:     do_stuff_when_it_doesnt_work() finally:     f.close() 

Yes, you gained two lines, but you added a level of nesting which doesn't make it easier to read. Is the purpose of the with statement to save you two lines or am I missing something?

It seems a lot to add a keyword just for that, so I feel like there is some syntax to handle the additional try/except that I don't know about.

Answer 1

For a start, it helps prevent the problem you've introduced in your try ... finally ... example.

The way you've structured it, if an exception is thrown while trying to open the file then you will never bind an open file to the name f, leading to either a NameError in the finally clause (if f has never been bound within scope) or something entirely unexpected (if it has).

The correct structure (equivalent to the with) is:

f = open(my_file)  try:     do_stuff_that_fails() finally:     f.close() 

(note - no need for an except clause if you've got nothing to do there).

Your second example similarly is wrong, and should be structured like:

try:     f = open(my_file)      try:         do_stuff_that_fails()     except EXPECTED_EXCEPTION_TYPES as e:         do_stuff_when_it_doesnt_work()     finally:         f.close()  except (IOError, OSError) as e:     do_other_stuff_when_it_we_have_file_IO_problems() 

The second is (as stated in another answer) that you can't forget to call f.close().

BTW, the term is "context management", not "resource management" - the with statement manages contexts, some of which may be resources, but others not. For example, it's also used with decimal to establish a decimal context for a particular block of code.

Finally (responding to your comment to the previous answer) you should never rely on refcount semantics for handling resources in Python. Jython, IronPython and PyPy all have non-refcount semantics, and there's nothing preventing CPython from going the other way (though it's highly unlikely for the immediate future). In a tight loop (e.g. os.walk) it is very very easy to run out of file handles if code relying on refcount semantics is run on a VM with different behaviour.

Answer 2

In the example you give, it's not better. It's best practice to catch exceptions as close to the point they're thrown to avoid catching unrelated exceptions of the same type.

try:     file = open(...) except OpenErrors...:     # handle open exceptions else:     try:         # do stuff with file     finally:         file.close() 

As unfortunately verbose as this is, the with statement doesn't allow you to catch exceptions thrown during its evaluation. There was a suggestion to add exception handling to this effect on the mailing list:

with open(...) as file:     # do stuff with file except OpenErrors...:     # handle open exceptions 

But this was shot down.

Finally it's worth noting that you can directly enter and exit context managers like so:

file = open(...).__enter__() file.__exit__(typ, val, tb) 

This is described in more detail here and here.

As a general guideline, with statements excel for cases where exceptions are not expected, and the default "enter/open/acquire" behaviour is adequate. Examples include required files, and simple locking.