Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Checking the next run time for scheduled periodic tasks in Celery (with Django)

*Using celery 3.1.25 because django-celery-beat 1.0.1 has an issue with scheduling periodic tasks.

Recently I encountered an issue with celerybeat whereby periodic tasks with an interval of a day or longer appear to be 'forgotten' by the scheduler. If I change the interval to every 5 seconds the task executes normally (every 5 seconds) and the last_run_at attribute gets updated. This means celerybeat is responding to the scheduler to a certain degree, but if I reset the last_run_at i.e. PeriodicTask.objects.update(last_run_at=None), none of the tasks with an interval of every day run anymore.

Celerybeat crashed at one point and that may have corrupted something so I created a new virtualenv and database to see if the problem persists. I'd like to know if there is a way to retrieve the next run time so that I don't have to wait a day to know whether or not my periodic task has been executed.

I have also tried using inspect <active/scheduled/reserved> but all returned empty. Is this normal for periodic tasks using djcelery's database scheduler?

Here's the function that schedules the tasks:

def schedule_data_collection(request, project):
    if (request.method == 'POST'):
        interval = request.POST.get('interval')

        target_project = Project.objects.get(url_path=project)    
        interval_schedule = dict(every=json.loads(interval), period='days')

        schedule, created = IntervalSchedule.objects.get_or_create(
            every=interval_schedule['every'],
            period=interval_schedule['period'],
        )

        task_name = '{} data collection'.format(target_project.name)

        try:
            task = PeriodicTask.objects.get(name=task_name)
        except PeriodicTask.DoesNotExist:
            task = PeriodicTask.objects.create(
                interval=schedule,
                name=task_name,
                task='myapp.tasks.collect_tool_data',
                args=json.dumps([target_project.url_path])
            )
        else:
            if task.interval != schedule:
                task.interval = schedule

            if task.enabled is False:
                task.enabled = True

            task.save()

        return HttpResponse(task.interval)
    else:
        return HttpResponseForbidden()
like image 823
FatHippo Avatar asked Mar 21 '17 10:03

FatHippo


People also ask

What is celery in Django?

Celery is a task queue with focus on real-time processing, while also supporting task scheduling.² There are two main usages of celery in a regular Django application.

How to add periodic tasks to existing celery app configuration?

Periodic tasks can be added to existing celery app configuration via “beat_schedule” options The celery beat can run using the below command. Make sure the celery worker is also running before starting the beat service.

How do I run periodic tasks in Django?

Running periodic tasks is an important part of web application development. For example, you might want to periodically check all users for late payments, and email a polite reminder to users that haven't paid. Django does not include any built-in functionality for handling periodic tasks. To perform periodic tasks, you'll need to rely on Celery.

How does celery-beat know when to run a task?

Now by looking at the periodic task instance in the DB, celery-beat will know to run the task when its time comes. We will now run our celery beat and worker processes to get the ball rolling.


1 Answers

You can see your scheduler by going into shell and looking at app.conf.CELERYBEAT_SCEDULE.

celery -A myApp shell
print(app.conf.CELERYBEAT_SCHEDULE)

This should show you all your Periodic Tasks.

like image 59
Christopher Whitehead Avatar answered Sep 19 '22 12:09

Christopher Whitehead