Prevent evaluating default function in dictionary.get or dictionary.setdefault for existing keys

Question

I'd like to keep track of key-value pairs I've processed already in a dictionary (or something else if it's better), where key is some input and value is the return output of some complex function/calculation. The main purpose is to prevent doing the same process over again if I wish to get the value for a key that has been seen before. I've tried using setdefault and get to solve this problem, but the function I call ends up getting executed regardless if the key exists in the dictionary.

Sample code:

def complex_function(some_key):
    """
    Complex calculations using some_key
    """
    return some_value

# Get my_key's value in my_dict. If my_key has not been seen yet,
# calculate its value and set it to my_dict[my_key]
my_value = my_dict.setdefault(my_key, complex_function(my_key))

complex_function ends up getting carried out regardless if my_key is in my_dict. I've also tried using my_dict.get(my_key, complex_function(my_key)) with the same result. For now, this is my fixed solution:

if my_key not in my_dict:
    my_dict[my_key] = complex_function(my_key)

my_value = my_dict[my_key]

Here are my questions. First, is using a dictionary for this purpose the right approach? Second, am I using setdefault correctly? And third, is my current fix a good solution to the problem? (I end up calling my_dict[my_key] twice if my_key doesn't exist)

Answer 1

So I went ahead and took Vincent's suggestion of using a decorator.

Here's what the new fix looks like:

import functools

@functools.lru_cache(maxsize=16)
def complex_function(some_input):
    """
    Complex calculations using some_input
    """
    return some_value

my_value = complex_function(some_input)

From what I understand so far, lru_cache uses a dictionary to cache the results. The key in this dictionary refers to argument(s) to the decorated function (some_input) and the value refers to the return value of the decorated function (some_value). So, if the function gets called with an argument that's previously been passed before, it would simply return the value referenced in the decorator's dictionary instead of running the function. If the argument hasn't been seen, the function proceeds as normal, and in addition, the decorator creates a new key-value pair in its dictionary.

I set the maxsize to 16 for now as I don't expect some_input to represent more than 10 unique values. One thing to note is that the arguments for the decorated function are required to be non-mutable and hashable, as it uses the arguments as keys for its dictionary.

Answer 2

Your current solution is fine. You are creating slightly more work, but significantly reducing the computational workload when the key is already present.

However, defaultdict is almost what you need here. By modifying it a little bit we can make it work exactly as you want.

from collections import defaultdict

class DefaultKeyDict(defaultdict):
    def __missing__(self, key):
        if self.default_factory is None:
            raise KeyError(key)
        self[key] = value = self.default_factory(key)
        return value

d = DefaultKeyDict(lambda key: key * 2)

assert d[1] == 2
print(d)