Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery with rabbitmq creates results multiple queues

I have installed Celery with RabbitMQ. Problem is that for every result that is returned, Celery will create in the Rabbit, queue with the task's ID in the exchange celeryresults.

I still want to have results, but on ONE queue.

my celeryconfig:

from datetime import timedelta
OKER_URL = 'amqp://'
CELERY_RESULT_BACKEND = 'amqp'
#CELERY_IGNORE_RESULT = True
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT=['json', 'application/json']
CELERY_TIMEZONE = 'Europe/Oslo'
CELERY_ENABLE_UTC = True

from celery.schedules import crontab

CELERYBEAT_SCHEDULE = {
    'every-minute': {
        'task': 'tasks.remote',
        'schedule': timedelta(seconds=30),
        'args': (),
    },
}

Is that possible? How?

Thanks!

like image 618
Himberjack Avatar asked Jan 08 '14 14:01

Himberjack


2 Answers

amqp backend creates a new queue for each task. Alternatively, there is a new rpc backend which keeps results in a single queue.

http://docs.celeryproject.org/en/master/whatsnew-3.1.html#new-rpc-result-backend

like image 195
mher Avatar answered Oct 21 '22 12:10

mher


Nothing unusual.

That is how celery works when we use amqp as result backend. It will create a new temporary queue for every result corresponding to each tasks that worker consumes.

If you are not interested in the result, you can try CELERY_IGNORE_RESULT = True setting

If you do want to store the result, then i would recommend using a different result backend like Redis.

like image 36
rohan Avatar answered Oct 21 '22 13:10

rohan