Python 3 __getattr__ behaving differently than in Python 2?

Question

I need to write a class that implements 32-bit unsigned integers the same way they work in C programming language. What I care about most are the binary shifts, but I generally want my class to:

1. Have the same interface int has and works with int properly
2. Any operation with my U32 class (int + U32, U32 + int etc) also return U32
3. Be pure-python - I don't want to use NumPy, ctypes, etc.

As can be found in this answer, I got a solution that works under Python 2. Recently I tried to run it under Python 3 and noticed that while the following test code works fine under older versions of Python, Python 3 raises an error:

class U32:
    """Emulates 32-bit unsigned int known from C programming language."""

    def __init__(self, num=0, base=None):
        """Creates the U32 object.

        Args:
            num: the integer/string to use as the initial state
            base: the base of the integer use if the num given was a string
        """
        if base is None:
            self.int_ = int(num) % 2**32
        else:
            self.int_ = int(num, base) % 2**32

    def __coerce__(self, ignored):
        return None

    def __str__(self):
        return "<U32 instance at 0x%x, int=%d>" % (id(self), self.int_)

    def __getattr__(self, attribute_name):
        print("getattr called, attribute_name=%s" % attribute_name)
        # you might want to take a look here:
        # https://stackoverflow.com/q/19611001/1091116
        r = getattr(self.int_, attribute_name)
        if callable(r):  # return a wrapper if integer's function was requested
            def f(*args, **kwargs):
                if args and isinstance(args[0], U32):
                    args = (args[0].int_, ) + args[1:]
                ret = r(*args, **kwargs)
                if ret is NotImplemented:
                    return ret
                if attribute_name in ['__str__', '__repr__', '__index__']:
                    return ret
                ret %= 2**32
                return U32(ret)
            return f
        return r

print(U32(4) / 2)
print(4 / U32(2))
print(U32(4) / U32(2))

And here's the error:

Traceback (most recent call last):
  File "u32.py", line 41, in <module>
    print(U32(4) / 2)
TypeError: unsupported operand type(s) for /: 'U32' and 'int'

It looks like the getattr trick doesn't get called at all in Python 3. Why is that? How can I get this code working both under Python 2 and 3?

Answer 1

Your Python 2 solution relied on old style class behaviour. Your Python 2 code would fail in the same manner as Python 3 were you to make your class inherit from object:

class U32(object):

This is because special methods are looked up on the type, not the object itself, for new-style classes. This behaviour change fixed several corner cases with the old model.

In practice this means that methods like __div__ are looked up directly on U32 itself, not as attributes on instances of U32, and the __getattr__ hook is not consulted.

Unfortunately, special method lookups also bypass any __getattr__ or __getattribute__ hooks. See the documentation on Special Method lookups:

In addition to bypassing any instance attributes in the interest of correctness, implicit special method lookup generally also bypasses the __getattribute__() method even of the object’s metaclass:

[...]

Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).

Your only option then, is to set all special methods dynamically on your class. A class decorator would do fine here:

def _build_delegate(name, attr, cls, type_):
    def f(*args, **kwargs):
        args = tuple(a if not isinstance(a, cls) else a.int_ for a in args)
        ret = attr(*args, **kwargs)
        if not isinstance(ret, type_) or name == '__hash__':
            return ret
        return cls(ret)
    return f

def delegated_special_methods(type_):
    def decorator(cls):
        for name, value in vars(type_).items():
            if (name[:2], name[-2:]) != ('__', '__') or not callable(value):
                continue
            if hasattr(cls, name) and not name in ('__repr__', '__hash__'):
                continue
            setattr(cls, name, _build_delegate(name, value, cls, type_))
        return cls
    return decorator

@delegated_special_methods(int)
class U32(object):
    def __init__(self, num=0, base=None):
        """Creates the U32 object.

        Args:
            num: the integer/string to use as the initial state
            base: the base of the integer use if the num given was a string
        """
        if base is None:
            self.int_ = int(num) % 2**32
        else:
            self.int_ = int(num, base) % 2**32
    def __coerce__(self, ignored):
        return None
    def __str__(self):
        return "<U32 instance at 0x%x, int=%d>" % (id(self), self.int_)

I updated the proxy function to handle multiple arguments correctly, and to auto-coerce back to your custom class if int is returned.