Memory-aware LRU caching in Python?

Question

I'm using Python 3's builtin functools.lru_cache decorator to memoize some expensive functions. I would like to memoize as many calls as possible without using too much memory, since caching too many values causes thrashing.

Is there a preferred technique or library for accomplishing this in Python?

For example, this question lead me to a Go library for system memory aware LRU caching. Something similar for Python would be ideal.

---

Note: I can't just estimate the memory used per value and set maxsize accordingly, since several processes will be calling the decorated function in parallel; a solution would need to actually dynamically check how much memory is free.

Answer 1

I ended up modifying the built-in lru_cache to use psutil.

The modified decorator takes an additional optional argument use_memory_up_to. If set, the cache will be considered full if there are fewer than use_memory_up_to bytes of memory available (according to psutil.virtual_memory().available). For example:

from .lru_cache import lru_cache

GB = 1024**3

@lru_cache(use_memory_up_to=(1 * GB))
def expensive_func(args):
    ...

Note: setting use_memory_up_to will cause maxsize to have no effect.

Here's the code: lru_cache.py