Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to apply functools.lru_cache to function with mutable parameters?

I have a function with one of the parameters as numpy.ndarray. It's mutable so it can not be cached by lru_cache.

Is there any existing solution for this?

like image 789
iuradz Avatar asked Jul 05 '15 22:07

iuradz


People also ask

What does Functools lru_cache do?

Python's functools module comes with the @lru_cache decorator, which gives you the ability to cache the result of your functions using the Least Recently Used (LRU) strategy. This is a simple yet powerful technique that you can use to leverage the power of caching in your code.

How do you use Functools in Python?

Functools module is for higher-order functions that work on other functions. It provides functions for working with other functions and callable objects to use or extend them without completely rewriting them. This module has two classes – partial and partialmethod.

How do you cache a function in Python?

lru_cache basics To memoize a function in Python, we can use a utility supplied in Python's standard library—the functools. lru_cache decorator. Now, every time you run the decorated function, lru_cache will check for a cached result for the inputs provided. If the result is in the cache, lru_cache will return it.

When would you use Lrucacache?

Any time you have a function where you expect the same results each time a function is called with the same inputs, you can use lru_cache. lru_cache only works for one python process. If you are running multiple subprocesses, or running the same script over and over, lru_cache will not work.


1 Answers

Possibly the simplest way of doing so is to memoize a version taking only immutable objects.

Say your function takes an np.array, and let's assume it's a 1d array. Fortunately, it is easily translated to a tuple:

import numpy as np

a = np.array([1, 2, 3, 4])
>> tuple(a)
(1, 2, 3, 4)

and vice versa:

>> np.array(tuple(a))
array([1, 2, 3, 4])

So you get something like

# Function called by the rest of your program
array_foo(a) # `a` is an `np.array`
    ...
    return tuple_foo(tuple(a))

then memoize instead this function:

# Internal implementation
@functools.lru_cache
tuple_foo(t) # `t` is a tuple
    ...
    a = np.array(t)
like image 184
Ami Tavory Avatar answered Nov 13 '22 14:11

Ami Tavory