Python - Launch a Long Running Process from a Web App

Question

I have a python web application that needs to launch a long running process. The catch is I don't want it to wait around for the process to finish. Just launch and finish.

I'm running on windows XP, and the web app is running under IIS (if that matters).

So far I tried popen but that didn't seem to work. It waited until the child process finished.

Answer 1

Ok, I finally figured this out! This seems to work:

from subprocess import Popen
from win32process import DETACHED_PROCESS

pid = Popen(["C:\python24\python.exe", "long_run.py"],creationflags=DETACHED_PROCESS,shell=True).pid
print pid
print 'done' 
#I can now close the console or anything I want and long_run.py continues!

Note: I added shell=True. Otherwise calling print in the child process gave me the error "IOError: [Errno 9] Bad file descriptor"

DETACHED_PROCESS is a Process Creation Flag that is passed to the underlying WINAPI CreateProcess function.

Answer 2

Instead of directly starting processes from your webapp, you could write jobs into a message queue. A separate service reads from the message queue and runs the jobs. Have a look at Celery, a Distributed Task Queue written in Python.