Django - gunicorn - App level variable (shared across workers)

Question

So I have a toy django + gunicorn project. I want to have a statistical model which is quite big loaded into memory only once and then get it reused in the workers/threads.

How/where do I define an app level variable? I tried putting it on settings.py, and also on wsgi.py

Answer 1

I don't think you can (nor should). Workers are separate processes that are forked before they run any of your code.

You could put the "model" (what is it that makes it big?) in a Redis DB and access it in each worker from there. The best option would probably be to create a separate service of which you run a single instance, and communicate with through HTTP or RPC from your worker (have a look at nameko for an easy (micro)services framework.

Another option would be to use a single Celery worker, and do the statistical calculations in a task.