Menu
I have scheduled task with celery beat to run every 3 hours:
'sync_stuff': {
'task': 'celery_tasks.sync_stuff',
'schedule': crontab(hour='*/3')
}
Sometimes it takes longer than 3 hours to finish the task and I want to ensure that celery does not schedule and run the task again while the old instance is still running.
Is there a way to do that just with celery or celerybeat settings?
619
For the delayed execution calls, you will see a shared_task instance instead of the result of the executed function. The primary reason for this is to allow the task to be run later, canceled later, or interrupted later. Later is the key phrase because the delayed execution task will now be run by the celery worker.
revoke cancels the task execution. If a task is revoked, the workers ignore the task and do not execute it. If you don't use persistent revokes your task can be executed after worker's restart. revoke has an terminate option which is False by default.
A task is just a Python function. You can think of scheduling a task as a time-delayed call to the function. For example, you might ask Celery to call your function task1 with arguments (1, 3, 3) after five minutes. Or you could have your function batchjob called every night at midnight.
Simplest way to check celery beat is running: ps aux | grep -i '[c]elerybeat' . If you get text string with pid it's running. Also you can make output of this command more pretty: ps aux | grep -i '[c]elerybeat' | awk '{print $2}' . If you get number - it's working, if you get nothing - it's not working.
Unfortunately, you have to implement a locking strategy yourself.
Read this part of the doc for additional details:
Like with cron, the tasks may overlap if the first task doesn’t complete before the next. If that’s a concern you should use a locking strategy to ensure only one instance can run at a time (see for example Ensuring a task is only executed one at a time).
Sources:
http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#entries http://docs.celeryproject.org/en/latest/tutorials/task-cookbook.html#cookbook-task-serial
143
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
