Middleware for celery tasks

Question

Currently we have a large number of celery tasks we use in our application, we have an exception which might raise on a large number of tasks.

We could create a try and catch block in each task and handle this specific exception accordingly, yet am searching for a way to catch any exception from any task (like a middleware line) for ease of maintenance.

Can any one advise?

We are using following versions:

celery==3.1.18
django-celery==3.1.16
Django==1.6.5

Answer 1

This can be done using a base abstract task handler. For exceptions in particular there is a on_failure handler.

from celery import Task

class MyBaseTask(Task):
    abstract = True

    def on_failure(self, exc, task_id, args, kwargs, einfo):
        # Task failed. What do you want to do?
        print('Task raised an exception: {}'.format(exc)

@app.task(base=MyBaseTask)
def my_task():
    # Your task code

See the Celery docs for more info on the available handlers and the arguments which are passed to each one: http://docs.celeryproject.org/en/latest/userguide/tasks.html#abstract-classes