Python metaclasses: Why isn't __setattr__ called for attributes set during class definition?

Question

I have the following python code:

class FooMeta(type):
    def __setattr__(self, name, value):
        print name, value
        return super(FooMeta, self).__setattr__(name, value)

class Foo(object):
    __metaclass__ = FooMeta
    FOO = 123
    def a(self):
        pass

I would have expected __setattr__ of the meta class being called for both FOO and a. However, it is not called at all. When I assign something to Foo.whatever after the class has been defined the method is called.

What's the reason for this behaviour and is there a way to intercept the assignments that happen during the creation of the class? Using attrs in __new__ won't work since I'd like to check if a method is being redefined.

Answer 1

A class block is roughly syntactic sugar for building a dictionary, and then invoking a metaclass to build the class object.

This:

class Foo(object):
    __metaclass__ = FooMeta
    FOO = 123
    def a(self):
        pass

Comes out pretty much as if you'd written:

d = {}
d['__metaclass__'] = FooMeta
d['FOO'] = 123
def a(self):
    pass
d['a'] = a
Foo = d.get('__metaclass__', type)('Foo', (object,), d)

Only without the namespace pollution (and in reality there's also a search through all the bases to determine the metaclass, or whether there's a metaclass conflict, but I'm ignoring that here).

The metaclass' __setattr__ can control what happens when you try to set an attribute on one of its instances (the class object), but inside the class block you're not doing that, you're inserting into a dictionary object, so the dict class controls what's going on, not your metaclass. So you're out of luck.

---

Unless you're using Python 3.x! In Python 3.x you can define a __prepare__ classmethod (or staticmethod) on a metaclass, which controls what object is used to accumulate attributes set within a class block before they're passed to the metaclass constructor. The default __prepare__ simply returns a normal dictionary, but you could build a custom dict-like class that doesn't allow keys to be redefined, and use that to accumulate your attributes:

from collections import MutableMapping


class SingleAssignDict(MutableMapping):
    def __init__(self, *args, **kwargs):
        self._d = dict(*args, **kwargs)

    def __getitem__(self, key):
        return self._d[key]

    def __setitem__(self, key, value):
        if key in self._d:
            raise ValueError(
                'Key {!r} already exists in SingleAssignDict'.format(key)
            )
        else:
            self._d[key] = value

    def __delitem__(self, key):
        del self._d[key]

    def __iter__(self):
        return iter(self._d)

    def __len__(self):
        return len(self._d)

    def __contains__(self, key):
        return key in self._d

    def __repr__(self):
        return '{}({!r})'.format(type(self).__name__, self._d)


class RedefBlocker(type):
    @classmethod
    def __prepare__(metacls, name, bases, **kwargs):
        return SingleAssignDict()

    def __new__(metacls, name, bases, sad):
        return super().__new__(metacls, name, bases, dict(sad))


class Okay(metaclass=RedefBlocker):
    a = 1
    b = 2


class Boom(metaclass=RedefBlocker):
    a = 1
    b = 2
    a = 3

Running this gives me:

Traceback (most recent call last):
  File "/tmp/redef.py", line 50, in <module>
    class Boom(metaclass=RedefBlocker):
  File "/tmp/redef.py", line 53, in Boom
    a = 3
  File "/tmp/redef.py", line 15, in __setitem__
    'Key {!r} already exists in SingleAssignDict'.format(key)
ValueError: Key 'a' already exists in SingleAssignDict

Some notes:

1. __prepare__ has to be a classmethod or staticmethod, because it's being called before the metaclass' instance (your class) exists.
2. type still needs its third parameter to be a real dict, so you have to have a __new__ method that converts the SingleAssignDict to a normal one
3. I could have subclassed dict, which would probably have avoided (2), but I really dislike doing that because of how the non-basic methods like update don't respect your overrides of the basic methods like __setitem__. So I prefer to subclass collections.MutableMapping and wrap a dictionary.
4. The actual Okay.__dict__ object is a normal dictionary, because it was set by type and type is finicky about the kind of dictionary it wants. This means that overwriting class attributes after class creation does not raise an exception. You can overwrite the __dict__ attribute after the superclass call in __new__ if you want to maintain the no-overwriting forced by the class object's dictionary.

---

Sadly this technique is unavailable in Python 2.x (I checked). The __prepare__ method isn't invoked, which makes sense as in Python 2.x the metaclass is determined by the __metaclass__ magic attribute rather than a special keyword in the classblock; which means the dict object used to accumulate attributes for the class block already exists by the time the metaclass is known.

Compare Python 2:

class Foo(object):
    __metaclass__ = FooMeta
    FOO = 123
    def a(self):
        pass

Being roughly equivalent to:

d = {}
d['__metaclass__'] = FooMeta
d['FOO'] = 123
def a(self):
    pass
d['a'] = a
Foo = d.get('__metaclass__', type)('Foo', (object,), d)

Where the metaclass to invoke is determined from the dictionary, versus Python 3:

class Foo(metaclass=FooMeta):
    FOO = 123
    def a(self):
        pass

Being roughly equivalent to:

d = FooMeta.__prepare__('Foo', ())
d['Foo'] = 123
def a(self):
    pass
d['a'] = a
Foo = FooMeta('Foo', (), d)

Where the dictionary to use is determined from the metaclass.