Python 3: super() raises TypeError unexpectedly

Question

Coming from Java, I'm struggling a bit getting down inheritance, abstract classes, static methods and similar concepts of OO programming in Python.

I have an implementation of an expression tree class, given (simplified) by

# Generic node class
class Node(ABC):
    @abstractmethod
    def to_expr(self):
        pass

    @staticmethod
    def bracket_complex(child):
        s = child.to_expr()
        return s if isinstance(child, Leaf) or isinstance(child, UnaryOpNode) else "(" + s + ")"


# Leaf class - used for values and variables
class Leaf(Node):
    def __init__(self, val):
        self.val = val

    def to_expr(self):
        return str(self.val)


# Unary operator node
class UnaryOpNode(Node):
    def __init__(self, op, child):
        self.op = op
        self.child = child

    def to_expr(self):
        return str(self.op) + super().bracket_complex(self.child)


# Binary operator node
class BinaryOpNode(Node):
    def __init__(self, op, lchild, rchild):
        self.op = op
        self.lchild = lchild
        self.rchild = rchild

    def to_expr(self):
        return super().bracket_complex(self.lchild) + " " + str(self.op) + " " + super().bracket_complex(self.rchild)


# Variadic operator node (arbitrary number of arguments)
# Assumes commutative operator
class VariadicOpNode(Node):
    def __init__(self, op, list_):
        self.op = op
        self.children = list_

    def to_expr(self):
        return (" " + str(self.op) + " ").join(super().bracket_complex(child) for child in self.children)

The method to_expr() works fine when called on instances of Leaf, UnaryOpNode and BinaryOpNode, but raises a TypeError when called on an instance of VariadicOpNode:

TypeError: super(type, obj): obj must be an instance or subtype of type

What am I doing wrong in that specific class that super() is suddenly not working?

In Java the static method would get inherited so I wouldn't even need the super call, but in Python this does not seem to be the case.

Answer 1

You're using super() without arguments in a generator expression. The super() is magic - it relies on information in the caller frame. Since the generator expression creates an additional function, super() without arguments does not work there. However since your superclass is not probable to change in the middle of execution of a method, you can move it out of the generator expression - this should also speed things up:

def to_expr(self):
    bracket_complex = super().bracket_complex
    return (" " + str(self.op) + " ").join(bracket_complex(child) for child in self.children)

However as static methods are "inherited" in Python, you could call the super method via self provided that you didn't override it in a subclass. Thus in this simple case you can write:

def to_expr(self):
    return (" " + str(self.op) + " ").join(self.bracket_complex(child) for child in self.children)

---

The implementation detail is that if no arguments are provided, the first argument shall be the value that is in the __class__ cell of the caller frame, and second shall be the first argument given to the caller function. Usually you just get a SystemError when using super in a wrong place, but generator expressions are wrapped inside an implicit generator function which creates another call frame. Unfortunately this function gets an argument, which leads the super() to complain with this exception.

So normally super() would be passed Foo there as the first argument, but within generator expression, a generator object was passed - and thus it is obvious that TypeError needs to be raised.

Answer 2

Answering your implied question:

In Java the static method would get inherited so I wouldn't even need the super call, but in Python this does not seem to be the case.

staticmethods are inherited:

class A:
    @staticmethod
    def a():
        print('Hello')

class B(A):
    def b(self):
        self.a()

b = B()
b.a()
b.b()

outputs:

Hello
Hello

Note that you cannot simply write:

class B(A):
    def b(self):
        a()

Python will never resolve a simple name to a method/staticmethod; for Python a() must be a function call, local or global. You must either reference the instance using self.a or the class using B.a.

In python the self is explicit as is the current class reference. Do not confuse with the implicit this of Java.