Django-Celery progress bar

Question

I use:

1. Celery
2. Django-Celery
3. RabbitMQ

I can see all my tasks in the Django admin page, but at the moment it has just a few states, like:

• RECEIVED
• RETRY
• REVOKED
• SUCCESS
• STARTED
• FAILURE
• PENDING

It's not enough information for me. Is it possible to add more details about a running process to the admin page? Like progress bar or finished jobs counter etc.

I know how to use the Celery logging function, but a GUI is better in my case for some reasons.

So, is it possible to send some tracing information to the Django-Celery admin page?

Answer 1

Here's my minimal progress-reporting Django backend using your setup. I'm still a Django n00b and it's the first time I'm messing with Celery, so this can probably be optimized.

from time import sleep

from celery import task, current_task
from celery.result import AsyncResult

from django.http import HttpResponse, HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.utils import simplejson as json
from django.conf.urls import patterns, url


@task()
def do_work():
    """ Get some rest, asynchronously, and update the state all the time """
    for i in range(100):
        sleep(0.1)
        current_task.update_state(state='PROGRESS',
            meta={'current': i, 'total': 100})


def poll_state(request):
    """ A view to report the progress to the user """
    if 'job' in request.GET:
        job_id = request.GET['job']
    else:
        return HttpResponse('No job id given.')

    job = AsyncResult(job_id)
    data = job.result or job.state
    return HttpResponse(json.dumps(data), mimetype='application/json')


def init_work(request):
    """ A view to start a background job and redirect to the status page """
    job = do_work.delay()
    return HttpResponseRedirect(reverse('poll_state') + '?job=' + job.id)


urlpatterns = patterns('webapp.modules.asynctasks.progress_bar_demo',
    url(r'^init_work$', init_work),
    url(r'^poll_state$', poll_state, name="poll_state"),
)

Answer 2

I am starting to try figuring this out myself. Start by defining a PROGRESS state exactly as explained on the Celery userguide, then all you need is to insert a js in your template that will update your progress bar.