Why does the class definition's metaclass keyword argument accept a callable?

Question

Background

The Python 3 documentation clearly describes how the metaclass of a class is determined:

• if no bases and no explicit metaclass are given, then type() is used
• if an explicit metaclass is given and it is not an instance of type(), then it is used directly as the metaclass
• if an instance of type() is given as the explicit metaclass, or bases are defined, then the most derived metaclass is used

Therefore, according to the second rule, it is possible to specify a metaclass using a callable. E.g.,

class MyMetaclass(type):
    pass

def metaclass_callable(name, bases, namespace):
    print("Called with", name)
    return MyMetaclass(name, bases, namespace)

class MyClass(metaclass=metaclass_callable):
    pass

class MyDerived(MyClass):
    pass

print(type(MyClass), type(MyDerived))

Question 1

Is the metaclass of MyClass: metaclass_callable or MyMetaclass? The second rule in the documentation says that the provided callable "is used directly as the metaclass". However, it seems to make more sense to say that the metaclass is MyMetaclass since

• MyClass and MyDerived have type MyMetaclass,
• metaclass_callable is called once and then appears to be unrecoverable,
• derived classes do not use (as far as I can tell) metaclass_callable in any way (they use MyMetaclass).

Question 2

Is there anything you can do with a callable that you can't do with an instance of type? What is the purpose of accepting an arbitrary callable?

Answer 1

Regarding your first question the metaclass should be MyMetaclass (which it's so):

In [7]: print(type(MyClass), type(MyDerived))
<class '__main__.MyMetaclass'> <class '__main__.MyMetaclass'>

The reason is that if the metaclass is not an instance of type python calls the methaclass by passing these arguments to it name, bases, ns, **kwds (see new_class) and since you are returning your real metaclass in that function it gets the correct type for metaclass.

And about the second question:

What is the purpose of accepting an arbitrary callable?

There is no special purpose, it's actually the nature of metaclasses which is because that making an instance from a class always calls the metaclass by calling it's __call__ method:

Metaclass.__call__()

Which means that you can pass any callable as your metaclass. So for example if you test it with a nested function the result will still be the same:

In [21]: def metaclass_callable(name, bases, namespace):
             def inner():
                 return MyMetaclass(name, bases, namespace)
             return inner()
   ....: 

In [22]: class MyClass(metaclass=metaclass_callable):
             pass
   ....: 

In [23]: print(type(MyClass), type(MyDerived))
<class '__main__.MyMetaclass'> <class '__main__.MyMetaclass'>

---

For more info here is how Python crates a class:

It calls the new_class function which it calls prepare_class inside itself, then as you can see inside the prepare_class python calls the __prepare__ method of the appropriate metaclass, beside of finding the proper meta (using _calculate_meta function ) and creating the appropriate namespace for the class.

So all in one here is the hierarchy of executing a metacalss's methods:

1. __prepare__ ¹
2. __call__
3. __new__
4. __init__

And here is the source code:

# Provide a PEP 3115 compliant mechanism for class creation
def new_class(name, bases=(), kwds=None, exec_body=None):
    """Create a class object dynamically using the appropriate metaclass."""
    meta, ns, kwds = prepare_class(name, bases, kwds)
    if exec_body is not None:
        exec_body(ns)
    return meta(name, bases, ns, **kwds)

def prepare_class(name, bases=(), kwds=None):
    """Call the __prepare__ method of the appropriate metaclass.

    Returns (metaclass, namespace, kwds) as a 3-tuple

    *metaclass* is the appropriate metaclass
    *namespace* is the prepared class namespace
    *kwds* is an updated copy of the passed in kwds argument with any
    'metaclass' entry removed. If no kwds argument is passed in, this will
    be an empty dict.
    """
    if kwds is None:
        kwds = {}
    else:
        kwds = dict(kwds) # Don't alter the provided mapping
    if 'metaclass' in kwds:
        meta = kwds.pop('metaclass')
    else:
        if bases:
            meta = type(bases[0])
        else:
            meta = type
    if isinstance(meta, type):
        # when meta is a type, we first determine the most-derived metaclass
        # instead of invoking the initial candidate directly
        meta = _calculate_meta(meta, bases)
    if hasattr(meta, '__prepare__'):
        ns = meta.__prepare__(name, bases, **kwds)
    else:
        ns = {}
    return meta, ns, kwds


def _calculate_meta(meta, bases):
    """Calculate the most derived metaclass."""
    winner = meta
    for base in bases:
        base_meta = type(base)
        if issubclass(winner, base_meta):
            continue
        if issubclass(base_meta, winner):
            winner = base_meta
            continue
        # else:
        raise TypeError("metaclass conflict: "
                        "the metaclass of a derived class "
                        "must be a (non-strict) subclass "
                        "of the metaclasses of all its bases")
    return winner

---

_{1. Note that it get called implicitly inside the new_class function and before the return.}

Answer 2

Well, the type is of course MyMetaClass. metaclass_callable is initially 'selected' as the metaclass since it's been specified in the metaclass kwarg and as such, it's __call__ (a simple function call) is going to be performed.

It just so happens that calling it will print and then invoke MyMetaClass.__call__ (which calls type.__call__ since __call__ hasn't been overridden for MyMetaClass). There the assignment of cls.__class__ is made to MyMetaClass.

metaclass_callable is called once and then appears to be unrecoverable

Yes, it is only initially invoked and then hands control over to MyMetaClass. I'm not aware of any class attribute that keeps that information around.

derived classes do not use (as far as I can tell) metaclass_callable in any way.

Nope, if no metaclass is explicitly defined, the best match for the metaclasses of bases (here MyClass) will be used (resulting in MyMetaClass).

---

As for question 2, pretty sure everything you can do with a callable is also possible by using an instance of type with __call__ overridden accordingly. As to why, you might not want to go full blown class-creation if you simply want to make minor changes when actually creating a class.