Is there any way to change the Celery config programmatically, after app init?

Question

I have set up a testing environment where I have Celery workers actually running in other processes, so that the full functionality of my system with Celery can be tested. This way, tasks actually run in a worker process and communicate back to the test runner, and so I don't need CELERY_ALWAYS_EAGER to test this functionality.

That being said, in some situations I have tasks which trigger off other tasks without caring when they finish, and I'd like to create tests which do - that is, to wait for those subtasks to finish. In these cases, the simplest approach seems to be to run just these tests eagerly (i.e. with CELERY_ALWAYS_EAGER set to true).

However, I don't see a straightforward way to change the config after Celery is initialized... and indeed, from a glance at the source code, it seems that it assumes the config won't change once the app starts.

This makes sense for a lot of options, since the worker would have to actually see the change, and changing it from the main program wouldn't do anything. But in the case of CELERY_ALWAYS_EAGER, this makes sense for the main program to be able to change it.

Is there any straightforward/well-supported way to do this? If not, what's a preferably not-too-hacky way to do this?

Another option is to make the task in question return the task ids it started off, so that the test can then wait on them... but I don't like the idea of changing my API for the sole purpose of making it runnable in a unit test.

Answer 1

Simply changing variables on Celery's .conf object (an instance of Settings) works:

app.conf.CELERY_ALWAYS_EAGER = True

Although conf is indeed a @cached_property of Celery (in version 3.1.22 anyway), this caches the instance returned, not all the values - so the configuration is indeed dynamically updatable.