Python's Subprocess.Popen With Shell=True. Wait till it is completed

Question

Submitting a complex cmd string made of a full file path to an executable, the multiple flags, arguments, parameters, inputs and outputs seems to require me to set shell=True otherwise subprocess.Popen is not able understand anything more complex than just a simple path to executable (with no spaces in a filepath).

In my example I have quite a long cmd:

cmd = " '/Application/MyApp.app/Contents/MacOS/my_executable' '/Path/to/input/files' -some -flags -here -could -be -a -lot '/full/path/to/output/files' "

Submitting this cmd to subprocess.Popen " results to an error that complains on something about the path and not being able to find it.

So instead of using :

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

check_call seems workings quite well:

proc = subprocess.check_call(cmd, shell=True)

Interesting, only after shell is set to True

shell=True 

the subprocess.check_call works with a supplied cmd.

The side effect is that the rest of the code seems proceeds running without waiting for subprocess.check_call(cmd, shell=True) to finish first.

The code is designed the way that the rest of the execution is dependent on a result of subprocess.check_call(cmd, shell=True).

I wonder if there is anyway to enforce the code execution to wait till subprocess.check_call(cmd, shell=True) is finished. Thanks in advance!

Answer 1

As @mikkas suggest just use it as a list here is a working example:

mainProcess = subprocess.Popen(['python', pyfile, param1, param2], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# get the return value from the method
communicateRes = mainProcess.communicate()

stdOutValue, stdErrValue = communicateRes

You are calling python.exe pyfile param1 param2

By using communicate() you can get the stdout and stderr as a Tuple

You can use python method split() to split your string to a list for example:

cmd = "python.exe myfile.py arg1 arg2"

cmd.split(" ")

Output:

['python.exe', 'myfile.py', 'arg1', 'arg2']

Answer 2

I think the check_call function should wait for the command to finish.

See the docs here http://docs.python.org/2/library/subprocess.html

Answer 3

Check call does not wait. You need to do a process.wait() and check the return code explicitly to get the functionaly you want.

Process = subprocess.Popen('%s' %command_string,stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
            Process.wait()
            if Process1.returncode!=0:
                    print Process1.returncode
                    sendMail()
                    return
            else:
                    sendMail()