Lazy-loading variables using overloaded decorators

Question

I have a state object that represents a system. Properties within the state object are populated from [huge] text files. As not every property is accessed every time a state instance, is created, it makes sense to lazily load them.:

class State:
    def import_positions(self):
        self._positions = {}
        # Code which populates self._positions

    @property
    def positions(self):
        try:
            return self._positions
        except AttributeError:
            self.import_positions()
            return self._positions

    def import_forces(self):
        self._forces = {}
        # Code which populates self._forces

    @property
    def forces(self):
        try:
            return self._forces
        except AttributeError:
            self.import_forces()
            return self._forces

There's a lot of repetitive boilerplate code here. Moreover, sometimes an import_abc can populate a few variables (i.e. import a few variables from a small data file if its already open).

It makes sense to overload @property such that it accepts a function to "provide" that variable, viz:

class State:
    def import_positions(self):
        self._positions = {}
        # Code which populates self._positions

    @lazyproperty(import_positions)
    def positions(self):
        pass

    def import_forces(self):
        self._forces = {}
        # Code which populates self._forces and self._strain

    @lazyproperty(import_forces)
    def forces(self):
        pass

    @lazyproperty(import_forces)
    def strain(self):
        pass

However, I cannot seem to find a way to trace exactly what method are being called in the @property decorator. As such, I don't know how to approach overloading @property into my own @lazyproperty.

Any thoughts?

Answer 1

Maybe you want something like this. It's a sort of simple memoization function combined with @property.

def lazyproperty(func):
    values = {}
    def wrapper(self):
        if not self in values:
            values[self] = func(self)
        return values[self]
    wrapper.__name__ = func.__name__
    return property(wrapper)

class State:
    @lazyproperty
    def positions(self):
        print 'loading positions'
        return {1, 2, 3}

s = State()
print s.positions
print s.positions

Which prints:

loading positions
set([1, 2, 3])
set([1, 2, 3])

Caveat: entries in the values dictionary won't be garbage collected, so it's not suitable for long-running programs. If the loaded value is immutable across all classes, it can be stored on the function object itself for better speed and memory use:

try:
    return func.value
except AttributeError:
    func.value = func(self)
    return func.value