subprocess how to use command when shell = False

Question

I am able to run this code:

subprocess.call(" ./ApiStreamingClient.py -w flow-index -n admin -p admin localhost 1477389500000000000 1477389900000000000 | gzip - > out2.gz", shell=True)

but I would like to run it with option shell=False in two steps. How can I do that? I'm not able to find any simple example.

I think I need to split the command in two parts separated by | but I could not understand how.

Answer 1

I'd do it this way:

import gzip
import subprocess
import sys

command = [
    sys.executable,
    'ApiStreamingClient.py',
    '-w', 'flow-index',
    '-n', 'admin',
    '-p', 'admin',
    'localhost',
    '1477389500000000000',
    '1477389900000000000',
    ]
text = subprocess.check_output(command)
with gzip.open('out2.gz', 'w', 6) as outfile:
    outfile.write(text)

Note that without a shell, you can't just invoke a Python script as an executable--you need to invoke python directly and pass the script as an argument. But that's easy enough.

If you want to do the whole thing in streaming fashion (e.g. if text takes up too much memory), you can use stdout=subprocess.PIPE to get a handle to read from, and pass that to shutil.copyfileobj() instead of writing text all at once.