`super` in a `typing.NamedTuple` subclass fails in python 3.8

Question

I have code which worked in Python 3.6 and fails in Python 3.8. It seems to boil down to calling super in subclass of typing.NamedTuple, as below:

<ipython-input-2-fea20b0178f3> in <module>
----> 1 class Test(typing.NamedTuple):
      2     a: int
      3     b: float
      4     def __repr__(self):
      5         return super(object, self).__repr__()

RuntimeError: __class__ not set defining 'Test' as <class '__main__.Test'>. Was __classcell__ propagated to type.__new__?

In [3]: class Test(typing.NamedTuple): 
   ...:     a: int 
   ...:     b: float 
   ...:     #def __repr__(self): 
   ...:     #    return super(object, self).__repr__() 
   ...:                                                                         

>>> # works

The purpose of this super(object, self).__repr__ call is to use the standard '<__main__.Test object at 0x7fa109953cf8>' __repr__ instead of printing out all the contents of the tuple elements (which would happen by default). There are some questions on super resulting in similar errors but they:

1. Refer to the parameter-less version super()
2. Fail already in Python 3.6 (it worked for me before 3.6 -> 3.8 upgrade)
3. I fail to understand how to fix this anyway, given that it's not a custom metaclass I have control over but the stdlib-provided typing.NamedTuple.

My question is how can I fix this while maintaining backwards compatibility with Python 3.6 (otherwise I'd just use @dataclasses.dataclass instead of inheriting from typing.NamedTuple)?

A side question is how can this fail at definition time given that the offending super call is inside a method which is not even executed yet. For instance:

In [3]: class Test(typing.NamedTuple): 
   ...:     a: int 
   ...:     b: float 
   ...:     def __repr__(self): 
   ...:         return foo 

works (until we actually call the __repr__) even though foo is an undefined reference. Is super magical in that regard?

Answer 1

I was slightly wrong in the other question (which I just updated). Apparently, this behavior manifests in both cases of super. In hindsight, I should have tested this.

What's happening here is the metaclass NamedTupleMeta indeed doesn't pass __classcell__ over to type.__new__ because it creates a namedtuple on the fly and returns that. Actually, in Python's 3.6 and 3.7 (where this is still a DeprecationWarning), the __classcell__ leaks into the class dictionary since it isn't removed by NamedTupleMeta.__new__.

class Test(NamedTuple):
    a: int
    b: float
    def __repr__(self):
        return super().__repr__()

# isn't removed by NamedTupleMeta
Test.__classcell__
<cell at 0x7f956562f618: type object at 0x5629b8a2a708>

Using object.__repr__ directly as suggested by Azat does the trick.

how can this fail at definition time

The same way the following also fails:

class Foo(metaclass=1): pass

Many checks are performed while the class is being constructed. Among these, is checking if the metaclass has passed the __classcell__ over to type_new.

Answer 2

Unfortunately, I'm not so familiar with CPython internals and classes generation to say why it fails, but there is this CPython bug tracker issue which seems to be related and some words in Python docs

CPython implementation detail: In CPython 3.6 and later, the __class__ cell is passed to the metaclass as a __classcell__ entry in the class namespace. If present, this must be propagated up to the type.__new__ call in order for the class to be initialised correctly. Failing to do so will result in a RuntimeError in Python 3.8.

so probably somewhere during actual namedtuple creation we have a call to type.__new__ without __classcell__ propagated, but I don't know if it's the case.

But this particular case seems to be solvable by not using super() call with explicitly saying that "we need to have __repr__ method of the object class" like

class Test(typing.NamedTuple):
    a: int
    b: float
    __repr__ = object.__repr__