Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the difference between applying, running and calling of celery task?

Tags:

python

celery

What is the difference between mytask.apply(), mytask.run() and mytask()? Which way is preferable?

like image 645
lampslave Avatar asked Mar 28 '17 21:03

lampslave


People also ask

How do you call celery tasks?

If the task isn't registered in the current process you can use send_task() to call the task by name instead. So delay is clearly convenient, but if you want to set additional execution options you have to use apply_async .

Can celery task call another task?

To answer your opening questions: As of version 2.0, Celery provides an easy way to start tasks from other tasks. What you are calling "secondary tasks" are what it calls "subtasks".

What is difference between Apply_async and delay?

delay() has comes preconfigured and only requires arguments to be passed to the task — that's sufficient for most basic needs. Apply_async is more complex, but also more powerful then preconfigured delay. It is always better to use apply_async with specifically set options for maximum flexibility.

How to call a celery task from another app?

Also if you are sending task from a totally different app, working with the same Celery instance – you can't use import and thus can't do .delay or .apply_async For this case you can call the task by name. Let's see the example for the calling task from within the models.py

What are tasks in celery?

For development docs, go here . Tasks are the building blocks of Celery applications. A task is a class that can be created out of any callable. It performs dual roles in that it defines both what happens when a task is called (sends a message), and what happens when a worker receives that message.

What is shared_task in celery?

The "shared_task" decorator allows creation of Celery tasks for reusable apps as it doesn't need the instance of the Celery app. It is also easier way to define a task as you don't need to import the Celery app instance. Shared task has a lot of parameters and you have to decide which one to use depending on type of task.

Is it better to use signals or celery for web applications?

It might be easier to use signals, but in a larger context - web application pov, if you used email, chances are if the app grows big, you might need to do some additional background tasks, in which case , having one system (celery) would be easier. And celery handles these perfectly well..


2 Answers

You can find all of this in the celery package in celery/app/task.py, or in the documentation http://docs.celeryproject.org/en/latest/reference/celery.app.task.html

To elaborate in my own investigation:

  • run is the immediate (local blocking) form of delay with the same limitations (arguments get passed, however no access to other execution options), and is similar to calling. it will immediately return e.g. in case of an exception, so the call is fully blocking, and just like calling a function.
tasks.my_task.run(foo='bar')

my_task() is the documented one, and acts like run. (in fact, Task.__call__ executes self.run(*args, **kwargs)); It should be probably preferred over run(), as it does other things as well.

tasks.my_task(foo='bar')

so between mytask.run() and mytask() i would choose mytask(), except if i know i want run instead. If it would be rewritten, as true celery worker function, both would be rewritten as delay()

  • apply is the form which uses a syntax akin to apply_async. so your arguments are added as args/kwargs keywords:
tasks.my_task.apply(kwargs={'foo': 'bar'})

it is also the function called by apply_async if it is set to always_eager. A side effect is, that exceptions would continue execution, just as apply_async would, if it is not set to always_eager or gets the throw keyword set.

when i want to have a syntax which later gets changed into apply_async, i would prefer apply - generally i would prefer apply personally.

like image 54
g4borg Avatar answered Oct 26 '22 22:10

g4borg


In my understanding:

  • apply: is to call task and execute locally
  • run: never seen before
  • mytask(): just like call func

If you want to send message and execute remote,you should use apply_async or delay, referring call_task.

like image 37
Cheney Avatar answered Oct 26 '22 22:10

Cheney