IntEnum returning missing ints in _missing_ throws TypeError

Question

In Python 3.6 it was possible for a class inheriting from IntEnum to use the _missing_ function to return ints that, while not valid values, would not crash when parsing incoming data.

class ByteEnum(IntEnum):
    ENUM_VALUE=0

    @classmethod
    def _missing_(cls, value):
        if (0<=value<=0xFF) and (value == int(value)):
            return value
        raise ValueError("%r is not a valid %s" % (value, cls.__name__))

When calling ByteEnum(13) i would expect to receive 13. This worked in python 3.6, but in 3.7 it throws a TypeError.

What can I do to have the ByteEnum have the same behaviour it had in Python 3.6, where any integer specified in the _missing_ function would be allowed. Preferably without having to specify all missing values, or reverting to Python 3.6.

log:

>>> import sys
>>> print(sys.version)
3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
>>> from enum import IntEnum
>>> class ByteEnum(IntEnum):
...     ENUM_VALUE=0
... 
...     @classmethod
...     def _missing_(cls, value):
...         if (0<=value<=0xFF) and (value == int(value)):
...             return value
...         raise ValueError("%r is not a valid %s" % (value, cls.__name__))
... 
>>> ByteEnum(13)
ValueError: 13 is not a valid ByteEnum

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\xx\AppData\Local\Programs\Python\Python37\lib\enum.py", line 310, in __call__
    return cls.__new__(cls, value)
  File "C:\Users\xx\AppData\Local\Programs\Python\Python37\lib\enum.py", line 564, in __new__
    raise exc
TypeError: error in ByteEnum._missing_: returned 13 instead of None or a valid member

I have been digging a little bit further and it looks like it is an intentional change: https://github.com/python/cpython/pull/9147. However this still leaves me with the question: what must I return from missing to make missing values work?

Answer 1

The new behavior fixes bpo-34536 and was backported into Python 3.7.2 rc1. The relevant commit is here. The issue leading to the bpo and the subsequent fix was brought up in the comments of this Q&A.

The old behavior did not do type checking on what _missing_ returns. In your case, as you state, it returns the integer value, whereas it should return the respective ByteEnum instance for that integer.

Following the recipe from the original implementation in Flag, you can make your enum behave in the intended way.

import sys
from enum import IntEnum

class ByteEnum(IntEnum):
    ENUM_VALUE = 0

    @classmethod
    def _missing_(cls, value):
        if isinstance(value, int) and 0 <= value <= 0xFF:
            return cls._create_pseudo_member_(value)
        return None # will raise the ValueError in Enum.__new__

    @classmethod
    def _create_pseudo_member_(cls, value):
        pseudo_member = cls._value2member_map_.get(value, None)
        if pseudo_member is None:
            new_member = int.__new__(cls, value)
            # I expect a name attribute to hold a string, hence str(value)
            # However, new_member._name_ = value works, too 
            new_member._name_ = str(value) 
            new_member._value_ = value
            pseudo_member = cls._value2member_map_.setdefault(value, new_member)
        return pseudo_member

print(sys.version)
print(ByteEnum(13))
print(ByteEnum(1337))

Output:

3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
ByteEnum.13    
Traceback (most recent call last):
[...]
ValueError: 1337 is not a valid ByteEnum

To get back to your expected behavior of your ByteEnum returning the value directly, you'd have to override __new__ to restore the old behavior.