*Using celery 3.1.25 because django-celery-beat 1.0.1 has an issue with scheduling periodic tasks.
Recently I encountered an issue with celerybeat whereby periodic tasks with an interval of a day or longer appear to be 'forgotten' by the scheduler. If I change the interval to every 5 seconds
the task executes normally (every 5 seconds) and the last_run_at
attribute gets updated. This means celerybeat is responding to the scheduler to a certain degree, but if I reset the last_run_at
i.e. PeriodicTask.objects.update(last_run_at=None)
, none of the tasks with an interval of every day
run anymore.
Celerybeat crashed at one point and that may have corrupted something so I created a new virtualenv and database to see if the problem persists. I'd like to know if there is a way to retrieve the next run time so that I don't have to wait a day to know whether or not my periodic task has been executed.
I have also tried using inspect <active/scheduled/reserved>
but all returned empty
. Is this normal for periodic tasks using djcelery's database scheduler?
Here's the function that schedules the tasks:
def schedule_data_collection(request, project):
if (request.method == 'POST'):
interval = request.POST.get('interval')
target_project = Project.objects.get(url_path=project)
interval_schedule = dict(every=json.loads(interval), period='days')
schedule, created = IntervalSchedule.objects.get_or_create(
every=interval_schedule['every'],
period=interval_schedule['period'],
)
task_name = '{} data collection'.format(target_project.name)
try:
task = PeriodicTask.objects.get(name=task_name)
except PeriodicTask.DoesNotExist:
task = PeriodicTask.objects.create(
interval=schedule,
name=task_name,
task='myapp.tasks.collect_tool_data',
args=json.dumps([target_project.url_path])
)
else:
if task.interval != schedule:
task.interval = schedule
if task.enabled is False:
task.enabled = True
task.save()
return HttpResponse(task.interval)
else:
return HttpResponseForbidden()
Celery is a task queue with focus on real-time processing, while also supporting task scheduling.² There are two main usages of celery in a regular Django application.
Periodic tasks can be added to existing celery app configuration via “beat_schedule” options The celery beat can run using the below command. Make sure the celery worker is also running before starting the beat service.
Running periodic tasks is an important part of web application development. For example, you might want to periodically check all users for late payments, and email a polite reminder to users that haven't paid. Django does not include any built-in functionality for handling periodic tasks. To perform periodic tasks, you'll need to rely on Celery.
Now by looking at the periodic task instance in the DB, celery-beat will know to run the task when its time comes. We will now run our celery beat and worker processes to get the ball rolling.
You can see your scheduler by going into shell and looking at app.conf.CELERYBEAT_SCEDULE.
celery -A myApp shell
print(app.conf.CELERYBEAT_SCHEDULE)
This should show you all your Periodic Tasks.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With