Invalidating a path from the Django cache recursively

Question

I am deleting a single path from the Django cache like this:

from models                   import Graph
from django.http              import HttpRequest
from django.utils.cache       import get_cache_key
from django.db.models.signals import post_save
from django.core.cache        import cache

def expire_page(path):
    request      = HttpRequest()
    request.path = path
    key          = get_cache_key(request)
    if cache.has_key(key):   
        cache.delete(key)

def invalidate_cache(sender, instance, **kwargs):
    expire_page(instance.get_absolute_url())

post_save.connect(invalidate_cache, sender = Graph)

This works - but is there a way to delete recursively? My paths look like this:

/graph/123
/graph/123/2009-08-01/2009-10-21

Whenever the graph with id "123" is saved, the cache for both paths needs to be invalidated. Can this be done?

Answer 1

You might want to consider employing a generational caching strategy, it seems like it would fit what you are trying to accomplish. In the code that you have provided, you would store a "generation" number for each absolute url. So for example you would initialize the "/graph/123" to have a generation of one, then its cache key would become something like "/GENERATION/1/graph/123". When you want to expire the cache for that absolute url you increment its generation value (to two in this case). That way, the next time someone goes to look up "/graph/123" the cache key becomes "/GENERATION/2/graph/123". This also solves the issue of expiring all the sub pages since they should be referring to the same cache key as "/graph/123".

Its a bit tricky to understand at first but it is a really elegant caching strategy which if done correctly means you never have to actually delete anything from cache. For more information here is a presentation on generational caching, its for Rails but the concept is the same, regardless of language.

Answer 2

Another option is to use a cache that supports tagging keys and evicting keys by tag. Django's built-in cache API does not have support for this approach. But at least one cache backend (not part of Django proper) does have support.

DiskCache* is an Apache2 licensed disk and file backed cache library, written in pure-Python, and compatible with Django. To use DiskCache in your project simply install it and configure your CACHES setting.

Installation is easy with pip:

$ pip install diskcache

Then configure your CACHES setting:

CACHES = {
    'default': {
        'BACKEND': 'diskcache.DjangoCache',
        'LOCATION': '/tmp/path/to/directory/',
    }
}

The cache set method is extended by an optional tag keyword argument like so:

from django.core.cache import cache

cache.set('/graph/123', value, tag='/graph/123')
cache.set('/graph/123/2009-08-01/2009-10-21', other_value, tag='/graph/123')

diskcache.DjangoCache uses a diskcache.FanoutCache internally. The corresponding FanoutCache is accessible through the _cache attribute and exposes an evict method. To evict all keys tagged with /graph/123 simply:

cache._cache.evict('/graph/123')

Though it may feel awkward to access an underscore-prefixed attribute, the DiskCache project is stable and unlikely to make significant changes to the DjangoCache implementation.

The Django cache benchmarks page has a discussion of alternative cache backends.

• Disclaimer: I am the original author of the DiskCache project.