How can I detect whether I'm running in a Celery worker?

Question

Is there a way to determine, programatically, that the current module being imported/run is done so in the context of a celery worker?

We've settled on setting an environment variable before running the Celery worker, and checking this environment variable in the code, but I wonder if there's a better way?

Answer 1

Simple,

import sys

IN_CELERY_WORKER_PROCESS = sys.argv and sys.argv[0].endswith('celery')\
    and 'worker' in sys.argv

if IN_CELERY_WORKER_PROCESS:
    print ('Im in Celery worker')

http://percentl.com/blog/django-how-can-i-detect-whether-im-running-celery-worker/

Answer 2

As of celery 4.2 you can also do this by setting a flag on the worker_ready signal

in celery.py:

from celery.signals import worker_ready
app = Celery(...)
app.running = False

@worker_ready.connect
def set_running(*args, **kwargs):
    app.running = True

Now you can check within your task by using the global app instance to see whether or not you are running. This can be very useful to determine which logger to use.

Answer 3

Depending on what your use-case scenario is exactly, you may be able to detect it by checking whether the request id is set:

@app.task(bind=True)
def foo(self):
    print self.request.id

If you invoke the above as foo.delay() then the task will be sent to a worker and self.request.id will be set to a unique number. If you invoke it as foo(), then it will be executed in your current process and self.request.id will be None.