Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django global data for threads

I have a shared global data object in my single-process multi-threaded django server - an object which is frequently used, but calculated infrequently. The calculation is time-consuming, so I want to share the results.

I thought it would work to use django's LocalMemCache for this simple data. Oddly, it seems to work for multiple ajax calls on a single page load, but for some reason, when I reload the page in my browser, the cache is empty again.

What am I doing wrong?

Is there a better way? Would a global variable be just as efficient, if I control write-access with a thread lock?

Here's basically what I'm doing:

from threading import Lock
from django.core.cache import get_cache

my_lock = Lock()
cache = get_cache('default')

def get_global_data():
    my_lock.acquire()
    try:
        cached_data = cache.get('data')
        if not cached_data:
            cached_data = big_function_to_calculate_data()
            cache.set('data', cached_data)
    finally:
        my_lock.release()
    return cached_data

# settings.py defines my django LocMemCache as:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'my_server'
    }
}

Edit:

The root cause of the problem was retrieving data based on an access control list (not part of the code here), which varied based on request type (GET, POST), etc. When calculating, this was a POST request with one set of access, and when reading it was a GET request with a different set of access, and was returning a different (and invalid) set of results.

like image 495
Adam Morris Avatar asked Oct 20 '22 08:10

Adam Morris


1 Answers

The above works. As a side note, using a persistent database cache seems to be preferred over LocMemCache.

# run python manage.py createcachetable

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'my_cache_table'
    }
}

The root cause of the problem was retrieving data based on an access control list, which varied based on request type (GET, POST), etc. When calculating, this was a POST request with one set of access, and when reading it was a GET request with a different set of access.

like image 167
Adam Morris Avatar answered Oct 23 '22 10:10

Adam Morris