Output from subprocess.Popen

Question

I have been writing some python code and in my code I was using "command"

The code was working as I intended but then I noticed in the Python docs that command has been deprecated and will be removed in Python 3 and that I should use "subprocess" instead.

"OK" I think, "I don't want my code to go straight to legacy status, so I should change that right now.

The thing is that subprocess.Popen seems to prepend a nasty string to the start of any output e.g.

<subprocess.Popen object at 0xb7394c8c>

All the examples I see have it there, it seems to be accepted as given that it is always there.

This code;

#!/usr/bin/python
import subprocess
output = subprocess.Popen("ls -al", shell=True)
print output

produces this;

<subprocess.Popen object at 0xb734b26c>
brettg@underworld:~/dev$ total 52
drwxr-xr-x  3 brettg brettg 4096 2011-05-27 12:38 .
drwxr-xr-x 21 brettg brettg 4096 2011-05-24 17:40 ..
<trunc>

Is this normal? If I use it as part of a larger program that outputs various formatted details to the console it messes everything up.

I'm using the command to obtain the IP address for an interface by using ifconfig along with various greps and awks to scrape the address.

Consider this code;

#!/usr/bin/python
import commands,subprocess

def new_get_ip (netif):
address = subprocess.Popen("/sbin/ifconfig " + netif + " | grep inet | grep -v inet6 | awk '{print $2}' | sed 's/addr://'i", shell=True)
return address

def old_get_ip (netif):
address = commands.getoutput("/sbin/ifconfig " + netif + " | grep inet | grep -v inet6 | awk '{print $2}' | sed 's/addr://'i")
return address

print "OLD IP is :",old_get_ip("eth0")
print ""
print "NEW IP is :",new_get_ip("eth0")

This returns;

brettg@underworld:~/dev$ ./IPAddress.py
OLD IP is : 10.48.16.60

NEW IP is : <subprocess.Popen object at 0xb744270c>
brettg@underworld:~/dev$ 10.48.16.60

Which is fugly to say the least.

Obviously I am missing something here. I am new to Python of course so I'm sure it is me doing the wrong thing but various google searches have been fruitless to this point.

What if I want cleaner output? Do I have to manually trim the offending output or am I invoking subprocess.Popen incorrectly?

Answer 1

The "ugly string" is what it should be printing. Python is correctly printing out the repr(subprocess.Popen(...)), just like what it would print if you said print(open('myfile.txt')).

Furthermore, python has no knowledge of what is being output to stdout. The output you are seeing is not from python, but from the process's stdout and stderr being redirected to your terminal as spam, that is not even going through the python process. It's like you ran a program someprogram & without redirecting its stdout and stderr to /dev/null, and then tried to run another command, but you'd occasionally see spam from the program. To repeat and clarify:

<subprocess.Popen object at 0xb734b26c>  <-- output of python program
brettg@underworld:~/dev$ total 52                     <-- spam from your shell, not from python
drwxr-xr-x  3 brettg brettg 4096 2011-05-27 12:38 .   <-- spam from your shell, not from python
drwxr-xr-x 21 brettg brettg 4096 2011-05-24 17:40 ..  <-- spam from your shell, not from python
...

In order to capture stdout, you must use the .communicate() function, like so:

#!/usr/bin/python
import subprocess
output = subprocess.Popen(["ls", "-a", "-l"], stdout=subprocess.PIPE).communicate()[0]
print output

Furthermore, you never want to use shell=True, as it is a security hole (a major security hole with unsanitized inputs, a minor one with no input because it allows local attacks by modifying the shell environment). For security reasons and also to avoid bugs, you generally want to pass in a list rather than a string. If you're lazy you can do "ls -al".split(), which is frowned upon, but it would be a security hole to do something like ("ls -l %s"%unsanitizedInput).split().

Answer 2

The variable output does not contain a string, it is a container for the subprocess.Popen() function. You don't need to print it. The code,

import subprocess
output = subprocess.Popen("ls -al", shell=True)

works perfectly, but without the ugly : <subprocess.Popen object at 0xb734b26c> being printed.