How do you tell if a context manager is reusable or reentrant?

Question

The Python contextlib docs note that context managers can be single-use, reusable, or reentrant. Reentrant ones can be used in more than one with statement including nested ones; reusable but not reentrant ones can be used in more than one with statement but not nested. A couple of examples are mentioned.

https://docs.python.org/3/library/contextlib.html#reentrant-context-managers

The docs for other context managers don't always mention what they are, though. For example, the docs for the patch context manager in unittest.mock don't mention this at all.

In general, what would you look at in the source to determine if a context manager is single use, reusable or reentrant?

Answer 1

One good way is to look at the object being returned/modifed or context being set up by the __enter__ call, and then see what happens to that object/context in the __exit__ call. Usually understanding how the state is actually being changed in each of those will make it obvious what will happen if you nest or re-use that object.

For example, when you with open("somefile") as f: you're getting a file handle back. In the __exit__, you're closing that file handle. Of course, it doesn't make sense to re-open a file handle object once its closed, nor does it make sense to open an already open file handle. And of course, closing the inner file handle will also close the outer file handle, which will be problematic. This is why no one ever does this:

f = open("file.txt")
with f:
  # stuff
# File will get closed here

There's no point in using f after it's closed, so we always use:

with open("file.txt") as f:

threading.Lock and threading.RLock objects can be used as context managers, too. Doing this makes sense:

l = threading.Lock()
with l: # This acquires the lock
    # stuff
# Lock got released

with l: # Acquired again
    # more stuff
# Released again

This does not, because Lock will deadlock if you try to take it recursively

l = threading.Lock()
with l:
    #  stuff
    with l: # Uh-oh, we tried acquiring an already acquired Lock. We'll deadlock here.

But that will work fine with RLock(), which can be acquired recursively.

Another example from the stdlib: multiprocessing.Pool() can be be used as a context manager in Python 3.3+. Here's what the docs say:

Pool objects now support the context manager protocol. __enter__() returns the pool object, and __exit__() calls terminate().

terminate() says it does this:

Stops the worker processes immediately without completing outstanding work. When the pool object is garbage collected terminate() will be called immediately.

Clearly, that's a one-time use.

The patch context manager is temporarily patching some object, and then undoing the patch when it's done. Nesting definitely doesn't make sense there - why would you want to re-patch something that's already patched? However, patching, unpatching, and then patching again does make sense logically, so it should be reusable (which testing shows is the case).

I don't think there is really any well-defined set of things that you can say "look for this and you'll know a context manager is reusable/re-entrant/one-time" because a context manager can literally do anything. The best you can do is understand what context is being established when in __enter__, how it's being broken down __exit__, and then logically determine what the implications of reusing/re-entering that context is.