Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can i use python coroutines as celery tasks

I have old celery version in my tornado project. My current celery version is 3.1.23. There was an old library https://github.com/mher/tornado-celery which is dead now, so i would like to get rid of it and update celery to new version.

I don't want to change a tons of code, so i would like to have an ability to use async/await syntax in my celery tasks. Is it possible to do something like this

@app.task
async def add(x, y):
    res = await some_async_function(x)
    ......

?

I've seen only examples with synchronous code and i'm not sure if my case is suitable for celery

Any tip will be good for asyncio or tornado.

UPD:

Even though celery tasks are already asynchronous, i need ability to use coroutines. I need coroutines, because currently we have a lot of tasks written with asynchronous calls, it was possible thanks to the tornado-celery library and some kind of black magic. We have database calls via async driver, async calls to AWS and so on. And because of that i need a possibility to work with asyncronous syntax inside celery tasks. I can use something like event_loop.run_until_complete(...) in asyncio or tornado.ioloop.IOLoop.instance().run_sync(lambda: do_something(1)) in tornado for example, but it's definitely not preferable way

like image 583
Alexandr Zayets Avatar asked Oct 15 '25 09:10

Alexandr Zayets


1 Answers

At the moment that is not possible as far as I know...

Celery tasks are executed asynchronously anyway, so really no need to use coroutines. delay(), apply_async() and send_task() are returning the AsyncResult (future-like object). This all makes sense as Celery predates Python's asyncio and async/await that later came into the language.

If you absolutely must use async/await, the only solution I can think of is to wrap your async logic inside a regular Celery task.

like image 183
DejanLekic Avatar answered Oct 16 '25 22:10

DejanLekic



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!