django 1.7 gotcha - django.setup() accidental recursion calls

Question

So, this one was a doozy. I am putting in a workaround, but just putting this out in case there is a better solution out there. And, having spent several hours before figuring this out, I'm also putting this out as a gotcha.

Basically, I was wondering if there are clever ways to avoid recursive calls to django.setup().

I have 3 or 4 batch scripts that I can run in either standalone mode or from celery. One of them is called build_profiles.py

The way celery gets to see them (in one of the tasks.py) files:

from pssecurity.batch.build_profiles import \
    ProfileManager as MgrCls_profiles, \
    getOptParser as getOptParser_profiles

In Django 1.6 this arrangement worked fine (I am not totally convinced celery is the best way to launch potentially stand-alone processes but that's another story).

When I tried to run build_profiles.py from the command line, it gave an AppRegistryNotReady error.

No problem, I thought, let's add the following to the top of build_profiles.py, as per https://docs.djangoproject.com/en/dev/ref/applications/#applications-troubleshooting

import django
django.setup()

And then nothing was working anymore with Django. unit tests would not run, manager.py runserver would hang. How could a change to a stand alone batch bring my system to a halt?

Turns out that django.setup() discovers celery which loads its tasks, and if one of those ends up doing its own django.setup()...

Answer 1

To build a bit on the example by @jl-peyret, I've used the following snippet to trigger the exception in the top of the file without having to wrap model access and know which one of the models will be accessed first:

from django.core.exceptions import AppRegistryNotReady
try:
    from django.apps import apps
    apps.check_apps_ready()
except AppRegistryNotReady:
    import django
    django.setup()

Answer 2

My workaround was to catch the AppRegistryNotReady error and call django.setup() only if required:

try:
    self.loadrdb(rdbname)
except AppRegistryNotReady:
    django.setup()
    self.loadrdb(rdbname)

this below also worked, but I think the try/catch is cleaner

if __name__ == "__main__":
    import django
    django.setup()

I wish they made idempotent, with django.setup() being smart enough to recognize that it had already been run and return quietly without doing any work.