Designing a Guava LoadingCache with variable entry expiry

Question

I am using Guava's LoadingCache into my project to handle thread-{safe,friendly} cache loading and it works wonderfully well. However, there is a limitation.

The current code defining the cache looks like this:

cache = CacheBuilder.newBuilder().maximumSize(100L).build(new CacheLoader<K, V>()
{
    // load() method implemented here
}

I don't specify an expiry time.

The problem is that according to the values of the key, some associated values may expire and others may not. And CacheLoader does not account for this, if you specify an expiry time, it is for each and every entry.

How would you tackle this problem?

Answer 1

Another alternative is my ExpiringMap (I'm the author), which supports variable entry expiration:

Map<String, String> map = ExpiringMap.builder().variableExpiration().build();
map.put("foo", "bar", ExpirationPolicy.ACCESSED, 5, TimeUnit.MINUTES);
map.put("baz", "pez", ExpirationPolicy.CREATED, 10, TimeUnit.MINUTES);

Answer 2

I suggest you to include expiration time directly to your entry class and manually evict it from cache if it is expired immediately after you fetched it from cache:

MyItem item = cache.getIfPresent(key);
if (item != null && item.isExpired()) {
    cache.invalidate(key);
    item = cache.get(key);
    // or use cache.put if you load it externally
}

As an alternative, I can suggest you to check EhCache library that supports per element expire policy.

Answer 3

LoadingCache offers some commonly used expiration policies, but when these fall short for what you need, you need to roll your own.

Just add a DelayQueue. Whenever you add something to the cache, add a Delayed to that queue, with the appropriate expiration time. The Delayed object should have a (weak?) reference to the key.

The final ingredient is that you need to periodically poll this queue, to see if something expired and has to be evicted. Don't necessarily add a thread to do this, you could just piggyback on whatever thread is accessing the LoadingCache. Just before accessing the cache, eg:

private void drainCache() {
  MyDelayed expired;
  while ((expired = delayedQueue.poll()) != null) {
    K key = expired.getReference();
    if (key != null) { // this only in case if you hold the key in a weak reference
      loadingCache.invalidate(key);
    }
  }
}

..
V lookup(K key) {
  drainCache();
  return loadingCache.getUnchecked(key);
}