class attribute lookup rule?

Question

>>> class D:
...     __class__ = 1
...     __name__ = 2
...
>>> D.__class__
<class 'type'>
>>> D().__class__
1
>>> D.__name__
'D'
>>> D().__name__
2

Why does D.__class__ return the name of the class, while D().__class__ returns the defined attribute in class D?

And from where do builtin attributes such as __class__ and __name__ come from?

I suspected __name__ or __class__ to be simple descriptors that live either in object class or somewhere, but this can't be seen.

In my understanding, the attribute lookup rule as follows in Python, omitting the conditions for descriptors etc..:

Instance --> Class --> Class.__bases__ and the bases of the other classes as well

Given the fact that a class is an instance of a metaclass, type in this case, why D.__class__ doesn't look for __class__ in D.__dict__?

Answer 1

The names __class__ and __name__ are special. Both are data descriptors. __name__ is defined on the type object, __class__ is defined on object (a base-class of all new-style classes):

>>> type.__dict__['__name__']
<attribute '__name__' of 'type' objects>
>>> type.__dict__['__name__'].__get__
<method-wrapper '__get__' of getset_descriptor object at 0x1059ea870>
>>> type.__dict__['__name__'].__set__
<method-wrapper '__set__' of getset_descriptor object at 0x1059ea870>
>>> object.__dict__['__class__']
<attribute '__class__' of 'object' objects>
>>> object.__dict__['__class__'].__get__
<method-wrapper '__get__' of getset_descriptor object at 0x1059ea2d0>
>>> object.__dict__['__class__'].__set__
<method-wrapper '__set__' of getset_descriptor object at 0x1059ea2d0>

Because they are data descriptors, the type.__getattribute__ method (used for attribute access on a class) will ignore any attributes set in the class __dict__ and only use the descriptors themselves:

>>> type.__getattribute__(Foo, '__class__')
<class 'type'>
>>> type.__getattribute__(Foo, '__name__')
'Foo'

Fun fact: type derives from object (everything in Python is an object) which is why __class__ is found on type when checking for data descriptors:

>>> type.__mro__
(<class 'type'>, <class 'object'>)

(type.__getattribute__(D, ...) is used directly as an unbound method, not D.__getattribute__(), because all special method access goes to the type).

See the Descriptor Howto an what constitutes a data descriptor and why that matters:

If an object defines both __get__() and __set__(), it is considered a data descriptor. Descriptors that only define __get__() are called non-data descriptors (they are typically used for methods but other uses are possible).

Data and non-data descriptors differ in how overrides are calculated with respect to entries in an instance’s dictionary. If an instance’s dictionary has an entry with the same name as a data descriptor, the data descriptor takes precedence. If an instance’s dictionary has an entry with the same name as a non-data descriptor, the dictionary entry takes precedence.

For data descriptors on type, a class is just another instance.

So when looking up the __class__ or __name__ attributes, it doesn't matter what is defined in the D.__dict__ namespace, because for either a data descriptor is found in the namespace formed by type and it's MRO.

These descriptors are defined in the typeobject.c C code:

static PyGetSetDef type_getsets[] = {
    {"__name__", (getter)type_name, (setter)type_set_name, NULL},
    /* ... several more ... */
}

/* ... */

PyTypeObject PyType_Type = {
    PyVarObject_HEAD_INIT(&PyType_Type, 0)
    "type",                                     /* tp_name */
    /* ... many type definition entries ... */
    type_getsets,                               /* tp_getset */
    /* ... many type definition entries ... */
}

/* ... */

static PyGetSetDef object_getsets[] = {
    {"__class__", object_get_class, object_set_class,
     PyDoc_STR("the object's class")},
    {0}
};

PyTypeObject PyBaseObject_Type = {
    PyVarObject_HEAD_INIT(&PyType_Type, 0)
    "object",                                   /* tp_name */
    /* ... many type definition entries ... */
    object_getsets,                             /* tp_getset */
    /* ... many type definition entries ... */
}

On instances, object.__getattribute__ is used, and it'll find the __name__ and __class__ entries in the D.__dict__ mapping before it'll find the data descriptors on object or type.

If you omit either, however, then looking up the names on D() will only __class__ as a data descriptor in the MRO of D (so, on object). __name__ is not found as the metatypes are not considered when resolving instance attributes.

As such you can set __name__ on an instance, but not __class__:

>>> class E: pass
...
>>> e = E()
>>> e.__class__
<class '__main__.E'>
>>> e.__name__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'E' object has no attribute '__name__'
>>> e.__dict__['__class__'] = 'ignored'
>>> e.__class__
<class '__main__.E'>
>>> e.__name__ = 'this just works'
>>> e.__name__
'this just works'