Dynamically reassign __len__ in a custom Python class

Question

It used to be possible to set internal functions like __len__() at runtime. Here is an example:

#! /usr/bin/python3

import sys

class FakeSequence:
    def __init__(self):
        self.real_sequence = list()
        self.append = self.real_sequence.append
        self.__len__ = self.real_sequence.__len__

    def workaround__len__(self):
        return len(self.real_sequence)

if __name__ == '__main__':
    fake_sequence = FakeSequence()
    fake_sequence.append(1)
    fake_sequence.append(2)
    fake_sequence.append(3)

    length = len(fake_sequence)
    sys.stdout.write("len(fake_sequence) is %d\n" % (length))

Here are the results when you try to run it:

$ python2 len_test
len(fake_sequence) is 3

$ python3 len_test
Traceback (most recent call last):
  File "len_test", line 18, in <module>
    length = len(fake_sequence)
TypeError: object of type 'FakeSequence' has no len()

If I define the __len__() method as part of the class (remove the 'workaround' above), it works as you would expect. If I define __len__() and reassign it as above FakeSequence.__len__() is called, it does not access the newly assigned __len__(), it always calls the FakeSequence class method.

Can you point me to documentation that would help explain why assigning instance methods for member functions no longer works? Note that assigning non-double-underscore methods still works fine. I can work around this easily enough, I'm more concerned that I missed something fundamental in the transition from Python 2 to Python 3. The behavior above is consistent with the Python 3 interpreters I have easy access to (3.4, 3.6, 3.7).

Answer 1

Magic methods are only looked up on classes, not on instances, as documented here. And it's also the case in Py2 for new-style classes (cf https://docs.python.org/2.7/reference/datamodel.html#special-method-lookup-for-new-style-classes).

I assume the main motivations is to cut down on lookups for better performances, but there might be other reasons, can't tell.

EDIT: actually, the motivations are clearly explained in the 2.7 doc:

The rationale behind this behaviour lies with a number of special methods such as hash() and repr() that are implemented by all objects, including type objects. If the implicit lookup of these methods used the conventional lookup process, they would fail when invoked on the type object itself:

Then:

Incorrectly attempting to invoke an unbound method of a class in this way is sometimes referred to as ‘metaclass confusion’, and is avoided by bypassing the instance when looking up special methods:

And finally:

In addition to bypassing any instance attributes in the interest of correctness, implicit special method lookup generally also bypasses the getattribute() method even of the object’s metaclass

Bypassing the getattribute() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods

So that's indeed mostly a performance optimization - which is not much of a surprise when you know about Python's attribute lookup mechanism and how Python's "methods" are implemented.