Using celery as a fault tolerant scheduler

Question

I would like to use celery w/ rabbitmq as a fault tolerant scheduler in a distributed environment. By fault tolerant, i mean that if a task is given to a worker and that worker goes down for whatever reason, celery should be able to reschedule it to another server. How is it possible to achieve this in an environment where there are multiple worker nodes?

Answer 1

Probably all you need is just to set CELERY_ACKS_LATE

Late ack means the task messages will be acknowledged after the task has been executed, not just before, which is the default behaviour. In this way if the worker crash rabbit MQ still have the message.

Here more info

Retry Lost or Failed Tasks (Celery, Django and RabbitMQ)