Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to configure Celery to send email alerts when tasks fail?

Tags:

celery

flower

How is it possible to configure celery to send email alerts when tasks are failing?

For example I want Celery to notify me when more than 3 tasks fail or more than 10 tasks are being retried.

Is it possible using celery or a utility (e.g. flower) or I have to write my own plugin?

like image 295
Ali Avatar asked Jul 07 '14 11:07

Ali


People also ask

What happens when a Celery task fails?

Celery will stop retrying after 7 failed attempts and raise an exception.

How does Celery execute tasks?

Celery workers are worker processes that run tasks independently from one another and outside the context of your main service. Celery beat is a scheduler that orchestrates when to run tasks. You can use it to schedule periodic tasks as well.


2 Answers

As far as I know, it's not possible out of the box.

You could write custom client on top of celery or flower or directly accessing RabbitMQ.

What I would do (and I am doing) is simply logging failed tasks and then use something like Graylog2 to monitor the log files, this works for all your infrastructure, not just Celery.

You can also use something like NewRelic which monitors your processes directly and offers many other features. Although email reporting on exceptions is somewhat limited in NewRelic.

A simple client/monitor probably is the quickest solution.

like image 35
Maiku Mori Avatar answered Oct 21 '22 22:10

Maiku Mori


Yes, all you need to do is set CELERY_SEND_TASK_ERROR_EMAILS = True and if Celery process fails django will send message with traceback to all emails set in ADMINS settings.

like image 189
daniula Avatar answered Oct 21 '22 23:10

daniula