Run a background task from a HTTP request

Question

I have some server code that I have unsuccessfully tried to run from a GET request, but I keep getting a timeout error. Is it possible to initiate a parallel thread during a GET request and then return a message to the user saying that their request is being handled in the background? If not what are some alternative ways my code can complete without giving the client side control?

I am a newbie running the latest Flask on a gunicorn server.

Answer 1

You can start a thread or a separate process to run your background task and then return a response back to the client. The response can include a reference to the background task that the client can use to poll for status or progress.

Have you heard of Celery? This is a task queue framework, it allows you to run one or more worker processes that execute asynchronous tasks requested by the application, in this case the Flask server.

I have written a small example that demonstrates how to combine Celery with Flask. The github repository for this example is: https://github.com/miguelgrinberg/flask-celery-example. In the README you can find a link to a blog post that I wrote that explains the project in detail.

Answer 2

Celery is definitely a great option, however, if your operation is simple and not critical you could consider asyncio. check this link