Python Flask returning a html page while simultaneously performing a function

Question

I'm currently creating a web app using Python Flask and I've run into a road block and I'm not sure if I'm even thinking about it correctly.

So my website's homepage is just a simple landing page with text input that is required to perform the websites function. What I am trying to accomplish is for the web app to perform two things after the text is input. First, the server takes the username input and performs a function that doesn't return anything to the user but creates a bunch of data that is logged into an sqlite database, and used later on in the process. Then, the server returns the web page for a survey that has to be taken after the username is input. However, the function that the server performs can take upwards of 2 minutes depending on the user. The way I currently have it coded, the server performs the function, then once it has finished, it returns the web page, so the user is stuck at a loading screen for up to 2 minutes.

@app.route("/survey")
def main(raw_user):
    raw_user = request.args.get("SteamID")      <
    games = createGameDict(user_obj)            <----- the function
    tag_lst = get_tags(games)                   <
    return render_template("survey_page.html")

Since the survey doesn't depend on the user input, instead of having the user sitting at a loading screen, I would like them to be able to start the survey while the functions works in the background, is that possible, and how would I do that?

Answer 1

Update: I've had to solve this problem a number of times in Flask, so I wrote a small Flask extension called Flask-Executor to do it for me. It's a wrapper for concurrent.futures that provides a few handy features, and is my preferred way of handling background tasks that don't require distribution in Flask.

---

For more complex background tasks, something like celery is your best bet. For simpler use cases however, what you want is the threading module.

Consider the following example:

from flask import Flask
from time import sleep

app = Flask(__name__)


def slow_function(some_object):
    sleep(5)
    print(some_object)

@app.route('/')
def index():
    some_object = 'This is a test'
    slow_function(some_object)
    return 'hello'

if __name__ == '__main__':
    app.run()

Here, we create a function, slow_function() that sleeps for five seconds before returning. When we call it in our route function it blocks the page load. Run the example and hit http://127.0.0.1:5000 in your browser, and you'll see the page wait five seconds before loading, after which the test message is printed in your terminal.

What we want to do is to put slow_function() on a different thread. With just a couple of additional lines of code, we can use the threading module to separate out the execution of this function onto a different thread:

from flask import Flask
from time import sleep
from threading import Thread

app = Flask(__name__)


def slow_function(some_object):
    sleep(5)
    print(some_object)

@app.route('/')
def index():
    some_object = 'This is a test'
    thr = Thread(target=slow_function, args=[some_object])
    thr.start()
    return 'hello'

if __name__ == '__main__':
    app.run()

What we're doing here is simple. We're creating a new instance of Thread and passing it two things: the target, which is the function we want to run, and args, the argument(s) to be passed to the target function. Notice that there are no parentheses on slow_function, because we're not running it - functions are objects, so we're passing the function itself to Thread. As for args, this always expects a list. Even if you only have one argument, wrap it in a list so args gets what it's expecting.

With our thread ready to go, thr.start() executes it. Run this example in your browser, and you'll notice that the index route now loads instantly. But wait another five seconds and sure enough, the test message will print in your terminal.

Now, we could stop here - but in my opinion at least, it's a bit messy to actually have this threading code inside the route itself. What if you need to call this function in another route, or a different context? Better to separate it out into its own function. You could make threading behaviour a part of slow function itself, or you could make a "wrapper" function - which approach you take depends a lot on what you're doing and what your needs are.

Let's create a wrapper function, and see what it looks like:

from flask import Flask
from time import sleep
from threading import Thread

app = Flask(__name__)


def slow_function(some_object):
    sleep(5)
    print(some_object)

def async_slow_function(some_object):
    thr = Thread(target=slow_function, args=[some_object])
    thr.start()
    return thr

@app.route('/')
def index():
    some_object = 'This is a test'
    async_slow_function(some_object)
    return 'hello'

if __name__ == '__main__':
    app.run()

The async_slow_function() function is doing pretty much exactly what we were doing before - it's just a bit neater now. You can call it in any route without having to rewrite your threading logic all over again. You'll notice that this function actually returns the thread - we don't need that for this example, but there are other things you might want to do with that thread later, so returning it makes the thread object available if you ever need it.