Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to keep a request context in a celery task, in Python Flask?

Is there a way to copy the request to a celery task in Flask in such a manner that the task executes inside the request context which initiated the task?
I need to access the flask security current user in a celery task, but since the task is outside the request context, I can not do that. I need additional information from the request, so just forwarding the current user to the task would not do the trick.

My task does inserts on the database. It needs the current user to save the id of the user which creates the row. Passing the user object to the task would solve the problem. However, the application logic is such that every insert/delete/update is logged via before flush event, which logs the user who did the modification, his IP, original url, the data it inserts...)

Log event is done like I said before flush, and it works in 99% scenarios. But when I have one lengthy task which I want to be a celery task, the request data is not available, nor is the current user (since it is outside the original request context)

like image 851
Mensur Avatar asked Jul 13 '15 12:07

Mensur


People also ask

What is request context in Flask?

The request context keeps track of the request-level data during a request. Rather than passing the request object to each function that runs during a request, the request and session proxies are accessed instead.

How does request work in Flask?

When the Flask application handles a request, it creates a Request object based on the environment it received from the WSGI server. Because a worker (thread, process, or coroutine depending on the server) handles only one request at a time, the request data can be considered global to that worker during that request.

Are celery tasks asynchronous?

What is Celery? “Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.” For this post, we will focus on the scheduling feature to periodically run a job/task.


1 Answers

The is no out-of-the-box way to pass the request or current_user objects to the celery task as they are non serializable. But people have worked around this by creating a wrapper to call the celery task in the request context.

The Celery task in a Flask request context blog post explores this topic in detail. The gists by Xion and derived one by aviaryan with the Request Context wrapper.

like image 130
Arunmozhi Avatar answered Oct 15 '22 06:10

Arunmozhi