subprocess.Popen and shlex.split formatting in windows and linux

Question

I am writing a wrapper to automate some android ADB shell commands via Python (2.7.2). Since, in some cases, I need to run the command asynchronously, I am using the subprocess.Popen method to issue shell commands.

I have run into a problem with the formatting of the [command, args] parameter of the Popen method, where there require command/args split is different between Windows and Linux:

# sample command with parameters
cmd = 'adb -s <serialnumber> shell ls /system'

# Windows:
s = subprocess.Popen(cmd.split(), shell=False) # command is split into args by spaces

# Linux:
s = subprocess.Popen([cmd], shell=False) # command is a list of length 1 containing whole command as single string

I have tried using shlex.split(), with and with out the posix flag:

# Windows
posix = False
print shlex.split(cmd, posix = posix), posix
# Linux
posix = True
print shlex.split(cmd, posix = posix), posix

Both cases return the same split.

Is there a method in subprocess or shlex that handles the OS-specific formats correctly?

This is my current solution:

import os
import tempfile
import subprocess
import shlex

# determine OS type
posix = False
if os.name == 'posix':
    posix = True

cmd = 'adb -s <serialnumber> shell ls /system'
if posix: # posix case, single command string including arguments
    args = [cmd]
else: # windows case, split arguments by spaces
    args = shlex.split(cmd)

# capture output to a temp file
o = tempfile.TemporaryFile()
s = subprocess.Popen(args, shell=False, stdout=o, stderr=o)
s.communicate()
o.seek(0)
o.read()
o.close()

I don't think shlex.split() is doing anything here, and cmd.split() achieves identical results.

Answer 1

They seem to function identically when I turn off shell=True

As per the docs:

On Unix, with shell=True: If args is a string, it specifies the command string to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

http://docs.python.org/library/subprocess.html

Answer 2

The shell=True argument tells it to have the command line evaluated by your shell, which on Windows will be Cmd.exe; on Linux, it'll likely be /bin/bash, but could also be some other related shell (zsh, tcsh, etc.). The difference in behavior is likely being caused by the shells interpreting the commands differently.

I'd strongly recommend not using shell=True if you can avoid it. Just something like this:

cmd = 'adb -s <serialnumber> shell ls /system'
s = subprocess.Popen(cmd.split())  # shell=False by default