How to design a latest recently used cache?

Question

How to design a latest recently used cache?

Suppose that you have visited some items. You need to design a data structure to hold these items. Each item is associated with the latest visited time.

Each time when you visit an item, check it in the data structure. If the item has been in the cache, update its visit time. Otherwise, insert it into the cache. The cache size is fixed, if it is full, delete the oldest item.

My solution:

1. Use a map < item, visitTime >

2. Initaliztion: Sort the map with f(visitTime) with descending order. O(nlg n)

3. If an item is visited, search it in the map with O(lg n).

4. If it has been in the map, update the time O(1). Sort the map O(lg n).

5. If not, insert it into map and then sort. O(lg n)

6. If map size > fixed size, delet the last element O(1).

Another solution:

1. Use hashtable < item , visitTime >

2. Sort it O(n lgn).

3. If an item is visited, search it in the talbe with O(1).

4. If it has been in the table , update the time O(1). Sort the table O(n lg n).

5. If not, insert it into table and then sort. O(n lg n)

6. If table size > fixed size, delet the last element O(1).

Are there better solutions ? O(n) ?

Answer 1

If you use a Doubly Linked List, you'll get O(1) insertion (after search), O(1) deletion, O(n) search.

Assuming you insert new items in the front:

If the cache is not full, just add to the front (O(1)).

If you need to update an item, find it (O(n)), remove it from the linked list (O(1)), then add to the front (O(1)).

If you need to delete the oldest to insert a new item, delete the end element (O(1)), and insert to the front (O(1)) [note: you need to search the list first in this case to see if the item is not already in the cache, so O(n)].

A Linked List can also give you the same time, since the search will leave you at the last element.

Answer 2

Python's LRU cache has O(1) insertion, deletion, and search. Its design uses a doubly-linked list of entries (arranged oldest-to-newest) and a hash table to locate a particular link.

Here's a simplified (but fast) version in under 40 lines of very basic Python. It shouldn't be hard to translate Python's solution into C++:

class LRU_Cache(object):

    def __init__(self, original_function, maxsize=1000):
        self.original_function = original_function
        self.maxsize = maxsize
        self.mapping = {}

        PREV, NEXT, KEY, VALUE = 0, 1, 2, 3
        self.head = [None, None, None, None]        # oldest
        self.tail = [self.head, None, None, None]   # newest
        self.head[NEXT] = self.tail

    def __call__(self, *key):
        PREV, NEXT, KEY, VALUE = 0, 1, 2, 3
        mapping, head, tail = self.mapping, self.head, self.tail
        sentinel = object()

        link = mapping.get(key, sentinel)
        if link is sentinel:
            value = self.original_function(*key)
            if len(mapping) >= self.maxsize:
                oldest = head[NEXT]
                next_oldest = oldest[NEXT]
                head[NEXT] = next_oldest
                next_oldest[PREV] = head
                del mapping[oldest[KEY]]
            last = tail[PREV]
            link = [last, tail, key, value]
            mapping[key] = last[NEXT] = tail[PREV] = link
        else:
            link_prev, link_next, key, value = link
            link_prev[NEXT] = link_next
            link_next[PREV] = link_prev
            last = tail[PREV]
            last[NEXT] = tail[PREV] = link
            link[PREV] = last
            link[NEXT] = tail
        return value

if __name__ == '__main__':
    p = LRU_Cache(ord, maxsize=3)
    for c in 'abcdecaeaa':
        print(c, p(c))