Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Clear all lru_cache in Python

I have functions in python that have caches with lru_cache e.g.

 @lru_cache(maxsize=None)
 def my_function():
    ...

While i can individually clear the caches with e.g. my_function.cache_clear() is there any way of clearing the caches of every function at once? [I was thinking that maybe there is a way of returning all function names loaded in memory, and then loop over them clearing the cache from each].

I'm specifically looking to implement as part of a fall-back, for cases where say 90% of the memory on my machine gets used.

like image 399
kyrenia Avatar asked Oct 26 '16 23:10

kyrenia


People also ask

What is Python lru_cache?

Python's functools module comes with the @lru_cache decorator, which gives you the ability to cache the result of your functions using the Least Recently Used (LRU) strategy. This is a simple yet powerful technique that you can use to leverage the power of caching in your code.

How do you cache a function in Python?

lru_cache basics To memoize a function in Python, we can use a utility supplied in Python's standard library—the functools. lru_cache decorator. Now, every time you run the decorated function, lru_cache will check for a cached result for the inputs provided. If the result is in the cache, lru_cache will return it.

How does LRU cache work?

A Least Recently Used (LRU) Cache organizes items in order of use, allowing you to quickly identify which item hasn't been used for the longest amount of time. Picture a clothes rack, where clothes are always hung up on one side. To find the least-recently used item, look at the item on the other end of the rack.


1 Answers

maybe there is a way of returning all function names loaded in memory, and then loop over them clearing the cache from each

Yes that is possible as well:

import functools
import gc

gc.collect()
wrappers = [
    a for a in gc.get_objects() 
    if isinstance(a, functools._lru_cache_wrapper)]

for wrapper in wrappers:
    wrapper.cache_clear()

I'm specifically looking to implement as part of a fall-back, for cases where say 90% of the memory on my machine gets used.

This is code you will probably not use under normal circumstances, but can be useful for debugging.

In my particular case I wanted to clear some dangling file handles from matplotlib (in order to focus on my own loose file handles), so it was not possible to use the solution by Alex Hall.

like image 139
BlackShift Avatar answered Sep 30 '22 04:09

BlackShift