Execute Subprocess in Background

Question

I have a python script which takes an input, formats it into a command which calls another script on the server, and then executes using subprocess:

import sys, subprocess

thingy = sys.argv[1]

command = 'usr/local/bin/otherscript.pl {0} &'.format(thingy)
command_list = command.split()
subprocess.call(command_list)

I append & to the end because otherscript.pl takes some time to execute, and I prefer to have run in the background. However, the script still seems to execute without giving me back control to the shell, and I have to wait until execution finishes to get back to my prompt. Is there another way to use subprocess to fully run the script in background?

Answer 1

& is a shell feature. If you want it to work with subprocess, you must specify shell=True like:

subprocess.call(command, shell=True)

This will allow you to run command in background.

Notes:

1. Since shell=True, the above uses command, not command_list.

2. Using shell=True enables all of the shell's features. Don't do this unless command including thingy comes from sources that you trust.

Safer Alternative

This alternative still lets you run the command in background but is safe because it uses the default shell=False:

p = subprocess.Popen(command_list)

After this statement is executed, the command will run in background. If you want to be sure that it has completed, run p.wait().