Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python celery - how to add CELERYBEAT_SCHEDULE task at runtime to a worker?

I have created a celery worker with a single celerybeat schedule task which runs at 5 seconds time interval. How can I add another beat task dynamically to the celery worker without stopping it?

Example

app.conf.update(
   CELERY_TASK_RESULT_EXPIRES=3600,
   CELERY_TIMEZONE = 'UTC',
   CELERYBEAT_SCHEDULE = {
    'long-run-5-secs': {
        'task': 'test_proj.tasks.test',
        'schedule': timedelta(seconds=5),
        'args': (16, )
    }
   }
)

With the above configuration, I am able to run the celery worker with beat mode successfully.

Now I need add the below beat schedule dynamically:

'long-run-2-secs': {
    'task': 'test_proj.tasks.test',
    'schedule': timedelta(seconds=2),
    'args': (14, ) },

Thanks

like image 331
muhil varnan Avatar asked Nov 13 '15 03:11

muhil varnan


1 Answers

I've been looking for solution for the very same problem. I am affraid you'll have to wait for Celery ver.4.0. Dynamic task scheduling is only currently supported in the development version : http://docs.celeryproject.org/en/master/userguide/periodic-tasks.html#beat-entries

like image 180
Jakub Czaplicki Avatar answered Sep 21 '22 22:09

Jakub Czaplicki