Memoizing all methods by default

Question

I’m writing an application that collects and displays the data from a scientific instrument. One of the pieces of data is a spectrum: essentially just a list of values, plus a dictionary with some metadata. Once the data has been collected by the application it does not change, so both the list and the metadata can be considered immutable.

I’d like to use this to my advantage by heavily memoizing the functions that perform calculations on the spectrum. Here’s a toy example:

class Spectrum(object):
    def __init__(self, values, metadata):
        self.values = values
        self.metadata = metadata
        # self.values and self.metadata should not change after this point.

    @property
    def first_value(self):
        return self.values[0]

    def multiply_by_constant(self, c):
        return [c*x for x in self.values]

    def double(self):
        return self.multiply_by_constant(2)

What I want is for each of these methods to be memoized by default. Is there some way (a metaclass?) to accomplish this without copying in one of these memoization decorators and writing @memoize everywhere?

Answer 1

I went ahead and wrote a metaclass to solve your question. It loops over all attributes and checks if they are callable (usually a function, method or class) and decorates those which are. Of course you would set decorator to your memoizing decorator (eg. functools.lru_cache).

If you only want to decorate the methods, and not any callable, you can replace the test hasattr(val, "__call__") with inspect.ismethod(val). But it could introduce a bug in the future where you don't remember it only works for methods, and add a function or class, which won't be memoized!

See this SO question for more info about metaclasses in Python.

def decorate(f):
    def wrap(*args, **kwargs):
        # Print a greeting every time decorated function is called
        print "Hi from wrap!"
        return f(*args, **kwargs)
    return wrap

class DecorateMeta(type):
    def __new__(cls, name, bases, dct):
        # Find which decorator to use through the decorator attribute
        try:
            decorator = dct["decorator"]
        except KeyError:
            raise TypeError("Must supply a decorator")

        # Loop over all attributes
        for key, val in dct.items():
            # If attribute is callable and is not the decorator being used
            if hasattr(val, "__call__") and val is not decorator:
                dct[key] = decorator(val)

        return type.__new__(cls, name, bases, dct)

class Test:
    __metaclass__ = DecorateMeta
    decorator = decorate

    def seasonal_greeting(self):
        print "Happy new year!"

Test().seasonal_greeting()

# Hi from wrap!
# Happy new year!

Answer 2

I adapted fridge’s answer into this:

from inspect import isfunction

class Immutable(type):
    def __new__(cls, name, bases, dct):
        for key, val in dct.items():
            # Look only at methods/functions; ignore those with
            # "special" names (starting with an underscore)
            if isfunction(val) and val.__name__[0] != '_':
                dct[key] = memoized(val)
        return type.__new__(cls, name, bases, dct)

The decorator is known ahead of time, so I don’t need to specify it in the object itself. I also only care about methods—although for reasons I don’t understand yet, all of the object’s methods are unbound when Immutable.__new__ sees them and so they’re functions, not methods. I also excluded methods with names starting with an underscore: in the case of memoizing, you don’t want to do anything to methods like __init__ or __eq__.