Why `type(x).__enter__(x)` instead of `x.__enter__()` in Python standard contextlib?

Question

In contextlib.py, I see the ExitStack class is calling __enter__() method via the type object (type(cm)) instead of direct method calls to the given object (cm).

I wonder why or why not.

e.g.,

• does it give better exception traces when an error occurs?
• is it just specific to some module author's coding style?
• does it have any performance benefits?
• does it avoid some artifacts/side-effects with complicated type hierarchies?

Answer 1

First of all, this is what happens when you do with something, it's not just contextlib that looks up special method on the type. Also, it's worth noting that the same happens with other special methods too: e.g. a + b results in type(a).__add__(a, b).

But why does it happen? This is a question that is often fired up on the python-dev and python-ideas mailing lists. And when I say "often", I mean "very often".

The last one were these: Missing Core Feature: + - * / | & do not call getattr and Eliminating special method lookup.

Here are some interesting points:

The current behaviour is by design - special methods are looked up as slots on the object's class, not as instance attributes. This allows the interpreter to bypass several steps in the normal instance attribute lookup process.

(Source)

It is worth noting that the behavior is even more magical than this. Even when looked up on the class, implicit special method lookup bypasses __getattr__ and __getattribute__ of the metaclass. So the special method lookup is not just an ordinary lookup that happens to start on the class instead of the instance; it is a fully magic lookup that does not engage the usual attribute-access-customization hooks at any level.

(Source)

This behavior is also documented on the reference documentation: Special method lookup, which says:

Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).

In short, performance is the main concern. But let's take a closer look at this.

What's the difference between type(obj).__enter__() and obj.__enter__()?

When you write obj.attr, type(obj).__getattribute__('attr') gets called. The default implementation of __getattribute__() looks for attr into the instance dictionary (i.e. obj.__dict__) and into the class namespace and, failing that, calls type(obj).__getattr__('attr').

Now, this was a quick explanation and I have omitted some details, however it should give you an idea of how complicated an attribute lookup can be, and how slow it can become. Short circuiting special method lookup surely provides performance improvements, as looking up obj.__enter__() in the "classical" way may be too slow.