Why doesn't my idea work in python2?

Question

Here is an idea for a dict subclass that can mutate keys. This is a simple self contained example that's just like a dict but is case insensitive for str keys.

from functools import wraps

def key_fix_decorator(f):
    @wraps(f)
    def wrapped(self, *args, **kwargs):
        if args and isinstance(args[0], str):
            args = (args[0].lower(),) + args[1:]
        return f(self, *args, **kwargs)
    return wrapped

class LowerDict(dict):
    pass

for method_name in '__setitem__', '__getitem__', '__delitem__', '__contains__', 'get', 'pop', 'setdefault':
    new_method = key_fix_decorator(getattr(LowerDict, method_name))
    setattr(LowerDict, method_name, new_method)

_{dev note: if you copy my code for your own uses, you should implement LowerDict.__init__ to check for any key collisions - I haven't bothered to include that for the purposes of this question}

On python3 it all seems to works fine:

>>> d = LowerDict(potato=123, spam='eggs')
>>> d['poTATo']
123
>>> d.pop('SPAm')
'eggs'
>>> d['A']
# KeyError: 'a'

In python2 it doesn't even import, here is the traceback:

  File "/tmp/thing.py", line 15, in <module>
    new_method = key_fix_decorator(getattr(LowerDict, method_name))
  File "/tmp/thing.py", line 4, in key_fix_decorator
    @wraps(f)
  File "/usr/lib/python2.7/functools.py", line 33, in update_wrapper
    setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'wrapper_descriptor' object has no attribute '__module__'

What could be the problem? I can't see any version-specific code except for the str/basestring thing, which is just a minor detail not a code-breaking issue.

Answer 1

The functools.wraps() version in Python 3 can handle function objects with some of the attributes it copies across missing; the one in Python 2 cannot. This is was because issue #3445 was fixed only for Python 3; the methods of dict are defined in C code and have no __module__ attribute.

Omitting the @wraps(f) decorator makes everything work in Python 2 too:

>>> def key_fix_decorator(f):
...     def wrapped(self, *args, **kwargs):
...         if args and isinstance(args[0], str):
...             args = (args[0].lower(),) + args[1:]
...         return f(self, *args, **kwargs)
...     return wrapped
... 
>>> class LowerDict(dict):
...     pass
... 
>>> for method_name in '__setitem__', '__getitem__', '__delitem__', '__contains__', 'get', 'pop', 'setdefault':
...     new_method = key_fix_decorator(getattr(LowerDict, method_name))
...     setattr(LowerDict, method_name, new_method)
... 
>>> d = LowerDict(potato=123, spam='eggs')
>>> d['poTATo']
123
>>> d.pop('SPAm')
'eggs'
>>> d['A']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in wrapped
KeyError: 'a'

You can replicate enough of what wraps does manually:

def key_fix_decorator(f):
    def wrapped(self, *args, **kwargs):
        if args and isinstance(args[0], str):
            args = (args[0].lower(),) + args[1:]
        return f(self, *args, **kwargs)
    wrapped.__name__ = f.__name__
    wrapped.__doc__ = f.__doc__
    return wrapped

or limit the attributes that wraps tries to copy across:

def key_fix_decorator(f):
    @wraps(f, assigned=('__name__', '__doc__'))
    def wrapped(self, *args, **kwargs):
        if args and isinstance(args[0], str):
            args = (args[0].lower(),) + args[1:]
        return f(self, *args, **kwargs)
    return wrapped

You don't really need to update __module__ attribute here; that is mostly useful only for introspection.