I have old celery version in my tornado project. My current celery version is 3.1.23. There was an old library https://github.com/mher/tornado-celery which is dead now, so i would like to get rid of it and update celery to new version.
I don't want to change a tons of code, so i would like to have an ability to use async/await syntax in my celery tasks. Is it possible to do something like this
@app.task
async def add(x, y):
res = await some_async_function(x)
......
?
I've seen only examples with synchronous code and i'm not sure if my case is suitable for celery
Any tip will be good for asyncio or tornado.
UPD:
Even though celery tasks are already asynchronous, i need ability to use coroutines. I need coroutines, because currently we have a lot of tasks written with asynchronous calls, it was possible thanks to the tornado-celery library and some kind of black magic. We have database calls via async driver, async calls to AWS and so on. And because of that i need a possibility to work with asyncronous syntax inside celery tasks. I can use something like event_loop.run_until_complete(...) in asyncio or tornado.ioloop.IOLoop.instance().run_sync(lambda: do_something(1)) in tornado for example, but it's definitely not preferable way
At the moment that is not possible as far as I know...
Celery tasks are executed asynchronously anyway, so really no need to use coroutines. delay()
, apply_async()
and send_task()
are returning the AsyncResult (future-like object). This all makes sense as Celery predates Python's asyncio and async/await that later came into the language.
If you absolutely must use async/await, the only solution I can think of is to wrap your async logic inside a regular Celery task.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With