Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using @functools.lru_cache with dictionary arguments

I have a method that takes (among others) a dictionary as an argument. The method is parsing strings and the dictionary provides replacements for some substrings, so it doesn't have to be mutable.

This function is called quite often, and on redundant elements so I figured that caching it would improve its efficiency.

But, as you may have guessed, since dict is mutable and thus not hashable, @functools.lru_cache can't decorate my function. So how can I overcome this?

Bonus point if it needs only standard library classes and methods. Ideally if it exists some kind of frozendict in standard library that I haven't seen it would make my day.

PS: namedtuple only in last resort, since it would need a big syntax shift.

like image 344
Evpok Avatar asked Jun 15 '11 13:06

Evpok


People also ask

What does Functools lru_cache do?

Python's functools module comes with the @lru_cache decorator, which gives you the ability to cache the result of your functions using the Least Recently Used (LRU) strategy. This is a simple yet powerful technique that you can use to leverage the power of caching in your code.

Is lru_cache thread safe?

lru_cache is a thread-safe LRU cache.

Is Functools a standard Python library?

Introduction. The functools module, part of Python's standard Library, provides useful features that make it easier to work with high order functions (a function that returns a function or takes another function as an argument ).

How do you cache a function in Python?

To memoize a function in Python, we can use a utility supplied in Python's standard library—the functools. lru_cache decorator. Now, every time you run the decorated function, lru_cache will check for a cached result for the inputs provided. If the result is in the cache, lru_cache will return it.


1 Answers

Instead of using a custom hashable dictionary, use this and avoid reinventing the wheel! It's a frozen dictionary that's all hashable.

https://pypi.org/project/frozendict/

Code:

from frozendict import frozendict  def freezeargs(func):     """Transform mutable dictionnary     Into immutable     Useful to be compatible with cache     """      @functools.wraps(func)     def wrapped(*args, **kwargs):         args = tuple([frozendict(arg) if isinstance(arg, dict) else arg for arg in args])         kwargs = {k: frozendict(v) if isinstance(v, dict) else v for k, v in kwargs.items()}         return func(*args, **kwargs)     return wrapped 

and then

@freezeargs @lru_cache def func(...):     pass 

Code taken from @fast_cen 's answer

Note: this does not work on recursive datastructures; for example, you might have an argument that's a list, which is unhashable. You are invited to make the wrapping recursive, such that it goes deep into the data structure and makes every dict frozen and every list tuple.

(I know that OP nolonger wants a solution, but I came here looking for the same solution, so leaving this for future generations)

like image 181
Cedar Avatar answered Sep 29 '22 20:09

Cedar