Spawning a separate thread of execution (i.e. sending log email to dev) in Flask Python?

Question

I am running a Flask webapp running behind uwsgi (2 processes). A part of my code involves pinging a remote resource, seeing if it has been modified (If-Modified-Since), and updating a local copy of that resource on the webserver if modified.

That update also sends myself an email of the diff. I'm concerned that this takes a long time, causing user requests to time out while sending the email.

Is the Python threading library the right way to tackle this? (spawn a thread and send the email there?) Will this interfere with uwsgi's processes at all?

Thanks for the help!

(on a side note: I am also a bit concerned about the 2 uwsgi processes bumping heads if they both try to update the resource on the local copy... I wonder if the threading module's lock capabilities is the right thing to look at for this problem as well?)

EDIT: To clarify, my primary concern is that the email task is part of the code execution. It takes a long time and runs before the return_template call, therefore holding up the response to the user. Is the Python threading library the right way to tackle this problem, given the Flask/uwsgi environment?

Answer 1

The best solution for this kind of tasks is using the uWSGI spooler. If you want to run the tasks at specific interval you can use the @timer decorator and let the spooler to the hard work:

from uwsgidecorators import *

#this will execute the_task() every 30 seconds in the spooler
@timer(30, target='spooler')
def the_task(signum)
    do_the_long_task()

Answer 2

If you try to make a server independent app (for instance, if you have in mind replacing uWSGI with Gunicorn later), I would recommend using Celery.

Here are the first steps: http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html