Who calls the metaclass

Question

This actually stems from a discussion here on SO.

Short version

def meta(name, bases, class_dict)
    return type(name, bases, class_dict)

class Klass(object):
    __metaclass__ = meta

meta() is called when Klass class declaration is executed.

Which part of the (python internal) code actually calls meta()?

Long version

When the class is declared, some code has to do the appropriate attribute checks and see if there is a __metaclass__ declared on a type. If such exists, it has to perform a method call on that metaclass with the well known (class_name, bases, class_dict) attributes. It is not really clear to me which code is responsible for that call.

I have done some digging in CPython (see below), but I would really like to have something closer to a definite answer.

Option 1: Called directly

The metaclass call is hardwired into the class parsing. If so, is there any evidence for this?

Option 2: It is called by type.__new__()

Code in type_call() calls type_new() which in turn calls _PyType_CalculateMetaclass(). This suggests that metaclass resolution is actually done during the call to type() when trying to find out which value to return from type()

This would be in in line with the notion that a "class" is a "callable that returns an object".

Option 3: Something different

All my guesses maybe completely wrong, of course.

Some example cases that we came up with in chat:

Example 1:

class Meta(type):
    pass

class A:
    __metaclass__ = Meta

A.__class__ == Meta

This is what Meta.__new__() returns, so this seems legit. The metaclass puts itself as A.__class__

Example 2:

class Meta(type):
    def __new__(cls, class_name, bases, class_dict):
        return type(class_name, bases, class_dict)

class A(object):
    __metaclass__ = Meta

A.__class__ == type

Edit 2: correct initial version, properly derive Meta from type.

Seems okay enough, but I'm not really sure this does what I think it does. Also: What is the canonical method to make it behave like in Example 1?

Edit 3: Using type.__new__(...) seems to work as expected, which also seems in favor of Option 2.

Can anybody with more in-depth knowledge of internal python magic enlighten me?

Edit: A for a quite concise primer on metaclasses: http://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/. Also has some really nice diagrams, references and also highlights the differences between python 2 and 3.

Edit 3: There is a good answer below for Python 3. Python 3 uses __build_class__ to create a class object. The code path is --however-- different in Python 2.

Answer 1

You can find the answer relatively easily. First, lets find the opcode for building a class.

>>> def f():
    class A(object):
        __metaclass__ = type

>>> import dis
>>> dis.dis(f)
  2           0 LOAD_CONST               1 ('A')
              3 LOAD_GLOBAL              0 (object)
              6 BUILD_TUPLE              1
              9 LOAD_CONST               2 (<code object A at 0000000001EBDA30, file "<pyshell#3>", line 2>)
             12 MAKE_FUNCTION            0
             15 CALL_FUNCTION            0
             18 BUILD_CLASS         
             19 STORE_FAST               0 (A)
             22 LOAD_CONST               0 (None)
             25 RETURN_VALUE    

So the opcode is BUILD_CLASS. Now let's search the source for that term (easily done on the github mirror).

You get a couple of results, but the most interesting of which is Python/ceval.c which declares the function static PyObject * build_class(PyObject *, PyObject *, PyObject *); and has a case statement for BUILD_CLASS. Search through the file and you can find the function definition of build_class starting at line 4430. And on line 4456 we find the bit of code you are looking for:

result = PyObject_CallFunctionObjArgs(metaclass, name, bases, methods,
                      NULL);

So the answer is the metaclass is resolved and called by the function that is responsible for executing the BUILD_CLASS opcode.

Answer 2

In Python 3, the metaclass is called in the code for the __build_class__ builtin function (which is called to handle class statements). This function is new in Python 3, and the equivalent C function build_class in Python 2 is not publicly exposed at the Python level. You can however find the source in python/ceval.c

Anyway, here's the relevant call to the metaclass object in the Python 3 __build_class__ implementation:

cls = PyEval_CallObjectWithKeywords(meta, margs, mkw);

The variable meta is the metaclass (either type or another metaclass found from an argument or from the type of a base class). margs is a tuple with the positional arguments (name, bases, dct) and mkw is a dictionary with the keyword arguments to the metaclass (a Python 3 only thing).

The Python 2 code does something similar:

result = PyObject_CallFunctionObjArgs(metaclass, name, bases, methods,
                                      NULL);