Is there an established memoize on-disk decorator for python?

Question

I have been searching a bit for a python module that offers a memoize decorator with the following capabilities:

• Stores cache on disk to be reused among subsequent program runs.
• Works for any pickle-able arguments, most importantly numpy arrays.
• (Bonus) checks whether arguments are mutated in function calls.

I found a few small code snippets for this task and could probably implement one myself, but I would prefer having an established package for this task. I also found incpy, but that does not seem to work with the standard python interpreter.

Ideally, I would like to have something like functools.lru_cache plus cache storage on disk. Can someone point me to a suitable package for this?

Answer 1

I don't know of any memoize decorator that takes care of all that, but you might want to have a look at ZODB. It's a persistence system built on top of pickle that provides some additional features including being able move objects from memory to disk when they aren't being used and the ability to save only objects that have been modified.

Edit: As a follow-up for the comment. A memoization decorator isn't supported out of the box by ZODB. However, I think you can:

• Implement your own persistent class
• Use a memoization decorator in the methods you need (any standard implementation should work, but it probably needs to be modified to make sure that the dirty bit is set)

After that, if you create an object of that class and add it to a ZODB database, when you execute one of the memoized methods, the object will be marked as dirty and changes will be saved to the database in the next transaction commit operation.