Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Communicate multiple times with a process without breaking the pipe?

It's not the first time I'm having this problem, and it's really bugging me. Whenever I open a pipe using the Python subprocess module, I can only communicate with it once, as the documentation specifies: Read data from stdout and stderr, until end-of-file is reached

proc = sub.Popen("psql -h darwin -d main_db".split(),stdin=sub.PIPE,stdout=sub.PIPE) print proc.communicate("select a,b,result from experiment_1412;\n")[0] print proc.communicate("select theta,zeta,result from experiment_2099\n")[0] 

The problem here is that the second time, Python isn't happy. Indeed, he decided to close the file after the first communicate:

Traceback (most recent call last): File "a.py", line 30, in <module>     print proc.communicate("select theta,zeta,result from experiment_2099\n")[0] File "/usr/lib64/python2.5/subprocess.py", line 667, in communicate     return self._communicate(input) File "/usr/lib64/python2.5/subprocess.py", line 1124, in _communicate      self.stdin.flush() ValueError: I/O operation on closed file 

Are multiple communications allowed?

like image 476
Manux Avatar asked Jun 17 '10 19:06

Manux


People also ask

What does subprocess pipe do?

In Python, the subprocess module allows you to execute linux/unix commands directly from Python. The official Python documentation is very useful would you need to go further.

What is the difference between subprocess call and run?

I can say that you use subprocess. call() when you want the program to wait for the process to complete before moving onto the next process. In the case of subprocess. run() , the program will attempt to run all the processes at once, inevitably causing the program to crash.

Does subprocess Popen block?

Popen is nonblocking. call and check_call are blocking. You can make the Popen instance block by calling its wait or communicate method.

What is Popen in Python?

Python method popen() opens a pipe to or from command. The return value is an open file object connected to the pipe, which can be read or written depending on whether mode is 'r' (default) or 'w'. The bufsize argument has the same meaning as in open() function.


2 Answers

I think you misunderstand communicate...

http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate

communicate sends a string to the other process and then waits on it to finish... (Like you said waits for the EOF listening to the stdout & stderror)

What you should do instead is:

proc.stdin.write('message')  # ...figure out how long or why you need to wait...  proc.stdin.write('message2') 

(and if you need to get the stdout or stderr you'd use proc.stdout or proc.stderr)

like image 110
Terence Honles Avatar answered Oct 07 '22 19:10

Terence Honles


I've had this problem before, and as far as I could ever figure, you couldn't do this with subprocess (which, I agree, is very counterintuitive if true). I ended up using pexpect (obtainable from PyPI).

like image 45
gilesc Avatar answered Oct 07 '22 19:10

gilesc