Using MongoDB as message queue for Celery

Question

I'm trying to use MongoDB as the message queue for Celery (in a Django app). The current development version of Celery (2.2.0rc2) is supposed to let you do this, but I can't seem to get any workers to pick up tasks I'm creating.

Versions: celery v2.2.0rc3
mongodb 1.6.5
pymongo 1.9
django-celery 2.2.0rc2

In my settings, I have:

CELERY_RESULT_BACKEND = "mongodb"
CELERY_MONGODB_BACKEND_SETTINGS = {
    # Shouldn't need these - defaults are correct.
    "host": "localhost",
    "port": 27017,
    "database": "celery",
    "taskmeta_collection": "messages",
}

BROKER_BACKEND = 'mongodb'
BROKER_HOST = "localhost"
BROKER_PORT = 27017
BROKER_USER = ""
BROKER_PASSWORD = ""
BROKER_VHOST = ""

import djcelery
djcelery.setup_loader()

I've created a test tasks.py file as follows:

from celery.decorators import task

@task()
def add(x, y):
    return x + y

If I fire up celeryd in the background, it appears to start normally. I then open a python shell and run the following:

>>> from myapp.tasks import add
>>> result = add.delay(5,5)
>>> result
<AsyncResult: 7174368d-288b-4abe-a6d7-aeba987fa886>
>>> result.ready()
False

Problem is that no workers ever pick up the tasks. Am I missing a setting or something? How do I point celery to the message queue?

Answer 1

We had this same issue. While the doc says all tasks should be registered in Celery by calling

import djcelery
djcelery.setup_loader()

it wasn't working properly. So, we still used the

CELERY_IMPORTS = ('YOUR_APP.tasks',) 

setting in settings.py. Also, make sure you restart Celery if you add a new task because Celery has to register the tasks when it first starts.

Django, Celerybeat and Celery with MongoDB as the Broker