Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

celery beat schedule: run task instantly when start celery beat?

If I create a celery beat schedule, using timedelta(days=1), the first task will be carried out after 24 hours, quote celery beat documentation:

Using a timedelta for the schedule means the task will be sent in 30 second intervals (the first task will be sent 30 seconds after celery beat starts, and then every 30 seconds after the last run).

But the fact is that in a lot of situations it's actually important that the the scheduler run the task at launch, But I didn't find an option that allows me to run the task immediately after celery starts, am I not reading carefully, or is celery missing this feature?

like image 655
timfeirg Avatar asked Mar 10 '15 10:03

timfeirg


2 Answers

I decide I could just declare a instance of every task, and execute them at celery launch. I don't like this at all because it makes starting celery beat extremely slow (if you have slow PeriodicTask), but it does what I want.

simply add this to the end of tasks.py:

########### spawn all tasks at launch! ############
localmess = locals().values()
for obj in localmess:
    if isclass(obj):
        if obj is not PeriodicTask and issubclass(obj, PeriodicTask):
            instance = obj()
            logger.info('running {0}'.format(obj))
            try:
                instance.run()
            except:
                logger.warn('task fail: {0}'.format(obj))
                pass

######## all tasks must be decleared above! ########
like image 60
timfeirg Avatar answered Sep 18 '22 17:09

timfeirg


You can run tasks right when the worker is ready with the worker_ready.connect decorator :

from celery.signals import worker_ready

@worker_ready.connect
def at_start(sender, **kwargs):
    """Run tasks at startup"""
    with sender.app.connection() as conn:
        sender.app.send_task("app.module.task", connection=conn)

Credits go to this answer : https://stackoverflow.com/a/14589445/3922534

like image 22
Mathieu Rollet Avatar answered Sep 22 '22 17:09

Mathieu Rollet