Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python subprocess module, how do I give input to the first of series of piped commands?

I am trying to use Python's subprocess module. What I require is to send input to the first process whose output becomes the input of the second process. The situation is basically almost the same as the example given in the documentation here: http://docs.python.org/library/subprocess.html#replacing-shell-pipeline except that I need to provide input the first command. Here is that example copied:

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

If we change the first line to:

p1 = Popen(["cat"], stdout=PIPE, stdin=PIPE)

How do I provide the input string to the process? If I attempt it by changing the final line to:

output = p2.communicate(input=inputstring)[0]

This doesn't work.

I do have a working version, which just stores the output of the first command in a string and then passes that to the second command. This isn't terrible as there is essentially no concurrency that can be exploited (in my actual use case the first command will exit rather quickly and produce all of its output at the end). Here is the working version in full:

import subprocess

simple = """Writing some text
with some lines in which the
word line occurs but others
where it does
not
"""

def run ():
  catcommand = [ "cat" ]
  catprocess = subprocess.Popen(catcommand,
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE)
  (catout, caterr) = catprocess.communicate(input=simple)
  grepcommand = [ "grep", "line" ]
  grepprocess = subprocess.Popen(grepcommand,
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE)
  (grepout, greperr) = grepprocess.communicate(input=catout)
  print "--- output ----"
  print grepout 
  print "--- error ----"
  print greperr 

if __name__ == "__main__":
  run()

I hope I've been clear enough, thanks for any help.

like image 472
Plan Avatar asked Feb 22 '11 15:02

Plan


People also ask

How do you pass command line arguments in subprocess?

You can pass a string to the command to be executed by subprocess. run method by using “input='string'” argument. As you can see, the code above passes “data. txt” as a string and not as a file object.

How do I input into subprocess?

write(input) with child_process as the previous result and input as a byte string to pass input to the subprocess through stdin . Call subprocess. communicate(input) with subprocess as the child process to get a tuple of the process's output and errors, if any.

What is Popen in subprocess?

The subprocess module defines one class, Popen and a few wrapper functions that use that class. The constructor for Popen takes arguments to set up the new process so the parent can communicate with it via pipes. It provides all of the functionality of the other modules and functions it replaces, and more.

How do you use a pipeline in subprocess?

To use a pipe with the subprocess module, you have to pass shell=True . In your particular case, however, the simple solution is to call subprocess. check_output(('ps', '-A')) and then str. find on the output.


1 Answers

If you do

from subprocess import Popen, PIPE
p1 = Popen(["cat"], stdout=PIPE, stdin=PIPE)

You should do p1.communicate("Your Input to the p1") and that will flow through the PIPE. The stdin is the process's input and you should communicate to that only.

The program which have given is absolutely fine, there seems no problem with that.

like image 66
Senthil Kumaran Avatar answered Sep 28 '22 19:09

Senthil Kumaran