Abstract Enum Class using ABCMeta and EnumMeta [duplicate]

Question

Simple Example

The goal is to create an abstract enum class through a metaclass deriving from both abc.ABCMeta and enum.EnumMeta. For example:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):
    pass

class A(abc.ABC):
    @abc.abstractmethod
    def foo(self):
        pass

class B(A, enum.IntEnum, metaclass=ABCEnumMeta):
    X = 1

class C(A):
    pass

Now, on Python3.7, this code will be interpreted without error (on 3.6.x and presumably below, it will not). In fact, everything looks great, our MRO shows B derived from both A and IntEnum.

>>> B.__mro__
(<enum 'B'>, __main__.A, abc.ABC, <enum 'IntEnum'>, int, <enum 'Enum'>, object)

Abstract Enum is not Abstract

However, even though B.foo has not been defined, we can still instantiate B without any issue, and call foo().

>>> B.X
<B.X: 1>
>>> B(1)
<B.X: 1>
>>> B(1).foo() 

This seems rather weird, since any other class that derives from ABCMeta cannot be instantiated, even if I use a custom metaclass.

>>> class NewMeta(type): 
...     pass
... 
... class AbcNewMeta(abc.ABCMeta, NewMeta):
...     pass
... 
... class D(metaclass=NewMeta):
...     pass
... 
... class E(A, D, metaclass=AbcNewMeta):
...     pass
...
>>> E()
TypeError: Can't instantiate abstract class E with abstract methods foo

Question

Why does a class using a metaclass derived from EnumMeta and ABCMeta effectively ignore ABCMeta, while any other class using a metaclass derived from ABCMeta use it? This is true even if I custom define the __new__ operator.

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):
    def __new__(cls, name, bases, dct):
        # Commented out lines reflect other variants that don't work
        #return abc.ABCMeta.__new__(cls, name, bases, dct)
        #return enum.EnumMeta.__new__(cls, name, bases, dct)
        return super().__new__(cls, name, bases, dct)

I'm rather confused, since this seems to fly in the face of what a metaclass is: the metaclass should define how the class is defined and behaves, and in this case, defining a class using a metaclass that is both abstract and an enumeration creates a class that silently ignores the abstract component. Is this a bug, is this intended, or is there something greater I am not understanding?

Answer 1

As stated on @chepner's answer, what is going on is that Enum metaclass overrides the normal metaclass' __call__ method, so that an Enum class is never instantiated through the normal methods, and thus, ABCMeta checking does not trigger its abstractmethod check.

However, on class creation, the Metaclass's __new__ is run normally, and the attributes used by the abstract-class mechanisms to mark the class as abstract do create the attribute ___abstractmethods__ on the created class.

So, all you have to do for what you intend to work, is to further customize your metaclass to perform the abstract check in the code to __call__:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):

    def __call__(cls, *args, **kw):
        if getattr(cls, "__abstractmethods__", None):
            raise TypeError(f"Can't instantiate abstract class {cls.__name__} "
                            f"with frozen methods {set(cls.__abstractmethods__)}")
        return super().__call__(*args, **kw)

This will make the B(1) expression to fail with the same error as abstractclass instantiation.

Note, however, that an Enum class can't be further inherited anyway, and it simply creating it without the missing abstractmethods may already violate what you want to check. That is: in your example above, class B can be declared and B.x will work, even with the missing foo method. If you want to prevent that, just put the same check in the metaclass' __new__:

import abc
import enum

class ABCEnumMeta(abc.ABCMeta, enum.EnumMeta):

    def __new__(mcls, *args, **kw):
        cls = super().__new__(mcls, *args, **kw)
        if issubclass(cls, enum.Enum) and getattr(cls, "__abstractmethods__", None):
            raise TypeError("...")
        return cls

    def __call__(cls, *args, **kw):
        if getattr(cls, "__abstractmethods__", None):
            raise TypeError(f"Can't instantiate abstract class {cls.__name__} "
                            f"with frozen methods {set(cls.__abstractmethods__)}")
        return super().__call__(*args, **kw)

(Unfortunatelly, the ABC abstract method check in CPython seems to be performed in native code, outside the ABCMeta.__call__ method - otherwise, instead of mimicking the error, we could just call ABCMeta.__call__ explicitly overriding super's behavior instead of hardcoding the TypeError there.)