Detect whether Celery is available and running

Question

I'm using Celery with Django for an online game.

I've written middleware to check whether Celery is available and running, based on this answer: Detect whether Celery is Available/Running

My code actually looks like this:

from celery.task.control import inspect

class CeleryCheckMiddleware(object):

    def process_request(self, request):
        insp = inspect().stats()
        if not insp:
           return render(...)
        else:
           return None

But I forgot the caveat in the comment at the bottom of that answer, 'I've discovered that the above adds two reply.celery.pidbox queues to rabbitmq every time it's run. This leads to an incremental increase in rabbitmq's memory usage.'

I'm now (only one day later!) noticing occasional 500 errors starting from the line insp = inspect().stats() and terminating with OSError: [Errno 4] Interrupted system call.

Is there a memory-safe way to check whether Celery is available and running?

Answer 1

This feels very heavy. You might be better off running an async task and collecting the result with an acceptable timeout. It's naive but it shouldn't impact resources a lot, depending on how often you have to call it thou .....

@app.job
def celery_alive():
    return "OK"

def process_request(self, request):
    res = celery_alive.apply_async()
    try:
        return "OK" == res.get(timeout=settings.ACCEPTABLE_TRANSACTION_TIME)
    except TimeoutError as e:
        return False