Calling __enter__ and __exit__ manually

Question

I've googled calling __enter__ manually but with no luck. So let's imagine I have MySQL connector class that uses __enter__ and __exit__ functions (originally used with with statement) to connect/disconnect from a database.

And let's have a class that uses 2 of these connections (for example for data sync). Note: this is not my real-life scenario, but it seems to be the simplest example.

Easiest way to make it all work together is class like this:

class DataSync(object):      def __init__(self):         self.master_connection = MySQLConnection(param_set_1)         self.slave_connection = MySQLConnection(param_set_2)      def __enter__(self):             self.master_connection.__enter__()             self.slave_connection.__enter__()             return self      def __exit__(self, exc_type, exc, traceback):             self.master_connection.__exit__(exc_type, exc, traceback)             self.slave_connection.__exit__(exc_type, exc, traceback)      # Some real operation functions  # Simple usage example with DataSync() as sync:     records = sync.master_connection.fetch_records()     sync.slave_connection.push_records(records) 

Q: Is it okay (is there anything wrong) to call __enter__/__exit__ manually like this?

Pylint 1.1.0 didn't issue any warnings on this, nor have I found any article about it (google link in the beggining).

And what about calling:

try:     # Db query except MySQL.ServerDisconnectedException:     self.master_connection.__exit__(None, None, None)     self.master_connection.__enter__()     # Retry 

Is this a good/bad practice? Why?

Answer 1

Your first example is not a good idea:

1. What happens if slave_connection.__enter__ throws an exception:

   • master_connection acquires its resource
   • slave_connection fails
   • DataSync.__enter__ propogates the exception
   • DataSync.__exit__ does not run
   • master_connection is never cleaned up!
   • Potential for Bad Things

2. What happens if master_connection.__exit__ throws an exception?

   • DataSync.__exit__ finished early
   • slave_connection is never cleaned up!
   • Potential for Bad Things

---

contextlib.ExitStack can help here:

def __enter__(self):     with ExitStack() as stack:         stack.enter_context(self.master_connection)         stack.enter_context(self.slave_connection)         self._stack = stack.pop_all()     return self  def __exit__(self, exc_type, exc, traceback):     self._stack.__exit__(self, exc_type, exc, traceback) 

Asking the same questions:

1. What happens if slave_connection.__enter__ throws an exception:

   • The with block is exited, and stack cleans up master_connection
   • Everything is ok!

2. What happens if master_connection.__exit__ throws an exception?

   • Doesn't matter, slave_connection gets cleaned up before this is called
   • Everything is ok!

3. Ok, what happens if slave_connection.__exit__ throws an exception?

   • ExitStack makes sure to call master_connection.__exit__ whatever happens to the slave connection
   • Everything is ok!

---

There's nothing wrong with calling __enter__ directly, but if you need to call it on more than one object, make sure you clean up properly!

Answer 2

Note: This answer doesn't properly account for possible failures when there are multiple calls to underlying __enter__ and __exit__ methods. See Eric's answer for one that does deal with that.

No, there's nothing wrong with that. There are even places in the standard library that do it. Like the multiprocessing module:

class SemLock(object):      def __init__(self, kind, value, maxvalue, *, ctx):             ...             try:                 sl = self._semlock = _multiprocessing.SemLock(                     kind, value, maxvalue, self._make_name(),                     unlink_now)             except FileExistsError:                 pass     ...      def __enter__(self):         return self._semlock.__enter__()      def __exit__(self, *args):         return self._semlock.__exit__(*args) 

Or the tempfile module:

class _TemporaryFileWrapper:      def __init__(self, file, name, delete=True):         self.file = file         self.name = name         self.delete = delete         self._closer = _TemporaryFileCloser(file, name, delete)      ...      # The underlying __enter__ method returns the wrong object     # (self.file) so override it to return the wrapper     def __enter__(self):         self.file.__enter__()         return self      # Need to trap __exit__ as well to ensure the file gets     # deleted when used in a with statement     def __exit__(self, exc, value, tb):         result = self.file.__exit__(exc, value, tb)         self.close()         return result 

The standard library examples aren't calling __enter__/__exit__ for two objects, but if you've got an object that's responsible for creating/destroying the context for multiple objects instead of just one, calling __enter__/__exit__ for all of them is fine.

The only potential gotcha is properly handling the return values of the __enter__ __exit__ calls for the objects you're managing. With __enter__, you need to make sure you're returning whatever state is required for the user of your wrapper object to get back from the with ... as <state>: call. With __exit__, you need to decide if you want to propagate any exception that occurred inside the context (by returning False), or suppress it (by returning True). Your managed objects could try to do it either way, you need to decide what makes sense for the wrapper object.