Why is the python subprocess output different from the shell?

Question

I am using the subprocess module to find out if a process is running. But the result is different when the process to find does not exist.

For example, in the shell, if the process python test.py does not exist, the output of ps -ef|grep python|grep test|awk '{print $2}' is empty. But in python:

cmd="ps -ef|grep python|grep test|awk '{print $2}'"
vp=subprocess.Popen(cmd,stdout=subprocess.PIPE,shell=True)
r=vp.communicate()[0]

The output r is not None. It is the pid of the shell executing cmd.

So how to get the desired result?

Answer 1

While the shell subprocess is running, its arguments are visible to ps since they are passed as a command line to sh.

shell=True works by invoking ['/bin/sh', '-c', cmdstring].

When you type a pipeline in the shell, each part of the pipeline is invoked separately so there is no process with both "python" and "test" in its arguments.

Your process tree looks like this:

python script.py
    /bin/sh -c "ps -ef|grep python|grep test|awk '{print $2}'"
        ps -ef
        grep python
        grep test
        awk '{print $2}'