Is Python *with* statement exactly equivalent to a try - (except) - finally block?

Question

I know this was widely discussed, but I still can't find an answer to confirm this: is the with statement identical to calling the same code in a try - (except) -finally block, where whatever one defines in the __exit__ function of the context manager is placed in the finally block?

For example -- are these 2 code snippets doing exactly the same thing?

import sys from contextlib import contextmanager  @contextmanager def open_input(fpath):     fd = open(fpath) if fpath else sys.stdin     try:         yield fd     finally:         fd.close()  with open_input("/path/to/file"):     print "starting to read from file..." 

the same as:

def open_input(fpath):     try:         fd = open(fpath) if fpath else sys.stdin         print "starting to read from file..."     finally:         fd.close()  open_input("/path/to/file") 

Thanks!

Answer 1

I'm going to put aside mentions of scope, because it's really not very relevant.

According to PEP 343,

with EXPR as VAR:     BLOCK 

translates to

mgr = (EXPR) exit = type(mgr).__exit__  # Not calling it yet value = type(mgr).__enter__(mgr) exc = True try:     try:         VAR = value  # Only if "as VAR" is present         BLOCK     except:         # The exceptional case is handled here         exc = False         if not exit(mgr, *sys.exc_info()):             raise         # The exception is swallowed if exit() returns true finally:     # The normal and non-local-goto cases are handled here     if exc:         exit(mgr, None, None, None) 

As you can see, type(mgr).__enter__ is called as you expect, but not inside the try.

type(mgr).__exit__ is called on exit. The only difference is that when there is an exception, the if not exit(mgr, *sys.exc_info()) path is taken. This gives with the ability to introspect and silence errors unlike what a finally clause can do.

---

contextmanager doesn't complicate this much. It's just:

def contextmanager(func):     @wraps(func)     def helper(*args, **kwds):         return _GeneratorContextManager(func, *args, **kwds)     return helper 

Then look at the class in question:

class _GeneratorContextManager(ContextDecorator):     def __init__(self, func, *args, **kwds):         self.gen = func(*args, **kwds)      def __enter__(self):         try:             return next(self.gen)         except StopIteration:             raise RuntimeError("generator didn't yield") from None      def __exit__(self, type, value, traceback):         if type is None:             try:                 next(self.gen)             except StopIteration:                 return             else:                 raise RuntimeError("generator didn't stop")         else:             if value is None:                 value = type()             try:                 self.gen.throw(type, value, traceback)                 raise RuntimeError("generator didn't stop after throw()")             except StopIteration as exc:                 return exc is not value             except:                 if sys.exc_info()[1] is not value:                     raise 

Unimportant code has been elided.

The first thing to note is that if there are multiple yields, this code will error.

This does not affect the control flow noticeably.

Consider __enter__.

try:     return next(self.gen) except StopIteration:     raise RuntimeError("generator didn't yield") from None 

If the context manager was well written, this will never break from what is expected.

One difference is that if the generator throws StopIteration, a different error (RuntimeError) will be produced. This means the behaviour is not totally identical to a normal with if you're running completely arbitrary code.

Consider a non-erroring __exit__:

if type is None:     try:         next(self.gen)     except StopIteration:         return     else:         raise RuntimeError("generator didn't stop") 

The only difference is as before; if your code throws StopIteration, it will affect the generator and thus the contextmanager decorator will misinterpret it.

This means that:

from contextlib import contextmanager  @contextmanager def with_cleanup(func):     try:         yield     finally:         func()  def good_cleanup():     print("cleaning")  with with_cleanup(good_cleanup):     print("doing")     1/0 #>>> doing #>>> cleaning #>>> Traceback (most recent call last): #>>>   File "", line 15, in <module> #>>> ZeroDivisionError: division by zero  def bad_cleanup():     print("cleaning")     raise StopIteration  with with_cleanup(bad_cleanup):     print("doing")     1/0 #>>> doing #>>> cleaning 

Which is unlikely to matter, but it could.

Finally:

else:     if value is None:         value = type()     try:         self.gen.throw(type, value, traceback)         raise RuntimeError("generator didn't stop after throw()")     except StopIteration as exc:         return exc is not value     except:         if sys.exc_info()[1] is not value:             raise 

This raises the same question about StopIteration, but it's interesting to note that last part.

if sys.exc_info()[1] is not value:     raise 

This means that if the exception is unhandled, the traceback will be unchanged. If it was handled but a new traceback exists, that will be raised instead.

This perfectly matches the spec.

---

TL;DR

• with is actually slightly more powerful than a try...finally in that the with can introspect and silence errors.

• Be careful about StopIteration, but otherwise you're fine using @contextmanager to create context managers.