How to make super() work by manually filling the __class__ cell?

Question

In Python 3 one can use super() instead of super(MyClass, self), but this only works in methods that were defined inside the class. As described in Michele Simionato's article the following example does not work:

def __init__(self):
    print('calling __init__')
    super().__init__()

class C(object):
    __init__ = __init__

if __name__ == '__main__':
    c = C()

It fails because super() looks for a __class__ cell, which is not defined in this case.

Is it possible to set this cell manually after the function has been defined, or is that impossible?

Unfortunately I don't understand how cells work in this context (didn't find much documentation for that). I'm hoping for something like

__init__.__class_cell_thingy__ = C

Of course I would only use this in a situation where the class assignment is unambiguous/unique (the whole process of adding methods to a class in is automatized in my case, so it would be simple to add such a line).

Answer 1

Seriously: you really don't want to do this.

But, it's useful for advanced users of Python to understand this, so I'll explain it.

Cells and freevars are the values assigned when a closure is created. For example,

def f():
    a = 1
    def func():
        print(a)
    return func

f returns a closure based on func, storing a reference to a. That reference is stored in a cell (actually in a freevar, but which one is up to the implementation). You can examine this:

myfunc = f()
# ('a',)
print(myfunc.__code__.co_freevars)
# (<cell at 0xb7abce84: int object at 0x82b1de0>,)
print(myfunc.__closure__)

("cells" and "freevars" are very similar. Freevars have names, where cells have indexes. They're both stored in func.__closure__, with cells coming first. We only care about freevars here, since that's what __class__ is.)

Once you understand that, you can see how super() actually works. Any function that contains a call to super is actually a closure, with a freevar named __class__ (which is also added if you refer to __class__ yourself):

class foo:
    def bar(self):
        print(__class__)

(Warning: this is where things get evil.)

These cells are visible in func.__closure__, but it's read-only; you can't change it. The only way to change it is to create a new function, which is done with the types.FunctionType constructor. However, your __init__ function doesn't have a __class__ freevar at all--so we need to add one. That means we have to create a new code object as well.

The below code does this. I added a base class B for demonstrative purposes. This code makes some assumptions, eg. that __init__ doesn't already have a free variable named __class__.

There's another hack here: there doesn't seem to be a constructor for the cell type. To work around that, a dummy function C.dummy is created which has the cell variable we need.

import types

class B(object):
    def __init__(self):
        print("base")

class C(B):
    def dummy(self): __class__

def __init__(self):
    print('calling __init__')
    super().__init__()

def MakeCodeObjectWithClass(c):
    """
    Return a copy of the code object c, with __class__ added to the end
    of co_freevars.
    """
    return types.CodeType(c.co_argcount, c.co_kwonlyargcount, c.co_nlocals,
            c.co_stacksize, c.co_flags, c.co_code, c.co_consts, c.co_names,
            c.co_varnames, c.co_filename, c.co_name, c.co_firstlineno,
            c.co_lnotab, c.co_freevars + ('__class__',), c.co_cellvars)

new_code = MakeCodeObjectWithClass(__init__.__code__)
old_closure = __init__.__closure__ or ()
C.__init__ = types.FunctionType(new_code, globals(), __init__.__name__,
    __init__.__defaults__, old_closure + (C.dummy.__closure__[0],))

if __name__ == '__main__':
    c = C()

Answer 2

Maybe, but by would you? In both cases you need to somehow be explicit of which class it is, because the implicit way didn't work. Maybe you can set the cell explicitly somehow, but there is no reason to do that. Just pass in the parameters explicitly.

def __init__(self):
    print('calling __init__')
    super(self.__class__, self).__init__()

class C(object):
    __init__ = __init__

if __name__ == '__main__':
    c = C()

(It's better if you can pass in the actual class directly, like so:

def __init__(self):
    print('calling __init__')
    super(C, self).__init__()

class C(object):
    __init__ = __init__

if __name__ == '__main__':
    c = C()

But if you can that, you could put the __init__ on C directly, so assume you can't.

Answer 3

You can use the function's dictionary.

def f(self):
    super(f.owner_cls, self).f()
    print("B")

def add_to_class(cls, member, name=None):
    if hasattr(member, 'owner_cls'):
        raise ValueError("%r already added to class %r" % (member, member.owner_cls))
    member.owner_cls = cls
    if name is None:
        name = member.__name__
    setattr(cls, name, member)

class A:
     def f(self):
         print("A")

class B(A):
     pass

add_to_class(B, f)

B().f()

You can even add another attribute member_name if you don't want to hardcode the name of the name of the member inside the function.