LRU cache on hard drive python [duplicate]

Question

I want to decorate a function as you would with functools.lru_cache with the results cached on the hard drive instead of memory. (The function's computing images from filenames.)

Answer 1

Here's some code to get you started:

from pathlib import Path
import pickle
import hashlib
import os

class LRU_Cache:

    def __init__(self, directory, original_function, maxsize=10):
        self.directory = directory
        self.original_function = original_function
        self.maxsize = maxsize
        try:
            os.mkdir(directory)
        except OSError:
            pass

    def __call__(self, *args):
        filename = hashlib.sha1(pickle.dumps(args)).hexdigest()
        fullname = os.path.join(self.directory, filename)
        try:
            with open(fullname, 'rb') as f:
                value = pickle.load(f)
            Path(fullname).touch()
            return value
        except FileNotFoundError:
            pass
        value = self.original_function(*args)
        with open(fullname, 'wb') as f:
            pickle.dump(value, f)
        filenames = os.listdir(self.directory)
        if len(filenames) <= self.maxsize:
            return
        fullnames = [os.path.join(self.directory, filename)
                     for filename in filenames]
        oldest = min(fullnames, key=lambda fn: os.stat(fn).st_mtime)
        os.remove(oldest)

It uses hashes the arguments to create a unique filename for each function call. The function return value is pickled using that filename.

Cache hits unpickle the stored result and update the file modification time.

If the cache directory exceeds a target size, the oldest cache file is removed.

Use it like this:

def square(x):
    print('!')
    return x ** 2

sqr = LRU_Cache('square_cache', square, 10)

Now call sqr normally and results will be cached to disk.