"implicit uses of special methods always rely on the class-level binding of the special method"

Question

I have difficulty understanding the last part (in bold) from Python in a Nutshell

Per-Instance Methods

An instance can have instance-specific bindings for all attributes, including callable attributes (methods). For a method, just like for any other attribute (except those bound to overriding descriptors), an instance-specific binding hides a class-level binding: attribute lookup does not consider the class when it finds a binding directly in the instance. An instance-specific binding for a callable attribute does not perform any of the transformations detailed in “Bound and Unbound Methods” on page 110: the attribute reference returns exactly the same callable object that was earlier bound directly to the instance attribute.

However, this does not work as you might expect for per-instance bindings of the special methods that Python calls implicitly as a result of various operations, as covered in “Special Methods” on page 123. Such implicit uses of special methods always rely on the class-level binding of the special method, if any. For example:

def fake_get_item(idx): return idx
class MyClass(object): pass
n = MyClass()
n.__getitem__ = fake_get_item
print(n[23])                      # results in:
# Traceback (most recent call last):
#   File "<stdin>", line 1, in ?
# TypeError: unindexable object

What does it mean specifically?

Why is the error of the example?

Thanks.

Answer 1

Neglecting all the fine details it basically says that special methods (as defined in Pythons data model - generally these are the methods starting with two underscores and ending with two underscores and are rarely, if ever, called directly) will never be used implicitly from the instance even if defined there:

n[whatever]  # will always call type(n).__getitem__(n, whatever)

This differs from attribute look-up which checks the instance first:

def fake_get_item(idx): 
    return idx

class MyClass(object): 
    pass

n = MyClass()
n.__getitem__ = fake_get_item
print(n.__getitem__(23))     # works because attribute lookup checks the instance first

---

There is a whole section in the documentation about this (including rationale): "Special method lookup":

3.3.9. Special method lookup

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:

>>> class C:
...     pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()

The rationale behind this behaviour lies with a number of special methods such as __hash__() and __repr__() that are implemented by all objects, including type objects. If the implicit lookup of these methods used the conventional lookup process, they would fail when invoked on the type object itself:

>>> 1 .__hash__() == hash(1)
True
>>> int.__hash__() == hash(int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor '__hash__' of 'int' object needs an argument

[...]

Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).

Answer 2

To put it even more plainly, it means that you can't redefine the dunder methods on the fly. As a consequence, ==, +, and the rest of the operators always mean the same thing for all objects of type T.

Answer 3

I'll try to summarize what the extract says and in particular the part in bold. Generally speaking, when Python tries to find the value of an attribute (including a method), it first checks the instance (i.e. the actual object you created), then the class. The code below illustrates the generic behavior.

class MyClass(object):
    def a(self):
        print("howdy from the class")

n = MyClass()
#here the class method is called
n.a()
#'howdy from the class'

def new_a():
    print("hello from new a")    
n.a = new_a
#the new instance binding hides the class binding
n.a()
#'hello from new a'

What the part in bold states is that this behavior does not apply to "Special Methods" such as __getitem__. In other words, overriding __getitem__ at the instance level (n.__getitem__ = fake_get_item in your exemple) does nothing : when the method is called through the n[] syntax, an error is raised because the class does not implement the method. (If the generic behavior also held in this case, the result of print(n[23]) would have been to print 23, i.e. executing the fake_get_item method).

Another example of the same behavior:

class MyClass(object):
    def __getitem__(self, idx):
        return idx
n = MyClass()
fake_get_item = lambda x: "fake"
print(fake_get_item(23))
#'fake'
n.__getitem__ = fake_get_item
print(n[23])
#'23'

In this example, the class method for __getitem__ (which returns the index number) is called instead of the instance binding (which returns 'fake').