Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to hash *args **kwargs for function cache?

Tags:

python

I'm working with xlwt which has a 4k limit on how many styles can be defined in an excel doc.

Normally, one creates styles like so:

style = xlwt.easyxf("font: bold 1")

Which I simply replaced with

def cached_easyxf(self, format):
    return self._cache.setdefault(format, xlwt.easyxf(format))

Which works perfectly. Now, I've found out that I need to pass in keyword arguments sometimes which got me thinking: how should I hash the args/kwargs signature?

Should I create a cache key based on str(value)? Pickle? What's most robust?

For my situation it looks like I can just convert key/values to strings and add it to my keys... but I'm now curious about a generic way to handle this with say unhashable types like arg=[1, 2, 3]

def cached_call(*args, **kwargs):
    return cache.get(what_here)
cached_call('hello')
cached_call([1, 2, 3], {'1': True})
like image 944
Yuji 'Tomita' Tomita Avatar asked Apr 19 '12 01:04

Yuji 'Tomita' Tomita


People also ask

What does * args and * Kwargs mean?

*args passes variable number of non-keyworded arguments and on which operation of the tuple can be performed. **kwargs passes variable number of keyword arguments dictionary to function on which operation of a dictionary can be performed.

How do I beat Kwarg?

Precede double stars ( ** ) to a dictionary argument to pass it to **kwargs parameter. Always place the **kwargs parameter at the end of the parameter list, or you'll get an error.

Does Python cache function result?

Memoization allows you to optimize a Python function by caching its output based on the parameters you supply to it. Once you memoize a function, it will only compute its output once for each set of parameters you call it with. Every call after the first will be quickly retrieved from a cache.

How do you beat Kwargs argument?

Kwargs allow you to pass keyword arguments to a function. They are used when you are not sure of the number of keyword arguments that will be passed in the function. Kwargs can be used for unpacking dictionary key, value pairs. This is done using the double asterisk notation ( ** ).


1 Answers

Here is the technique used in functools.lru_cache():

kwd_mark = object()     # sentinel for separating args from kwargs

def cached_call(*args, **kwargs):
    key = args + (kwd_mark,) + tuple(sorted(kwargs.items()))
    return cache.get(key)

Note, the above code handles keyword arguments but makes no attempt to handle non-hashable values like lists. Your idea for using the str or a list is a reasonable start. For set objects, you would need to sort the entries first, str(sorted(someset)). Other objects may not have a useful __repr__ or __str__ (i.e. they may show only the object type and location in memory). In summary, handling arbitrary unhashable arguments requires careful consideration of each object type.

like image 95
Raymond Hettinger Avatar answered Oct 06 '22 15:10

Raymond Hettinger