Is there any way to pass 'stdin' as an argument to another process in python?

Question

I'm trying to create a script which is using multiprocessing module with python. The script (lets call it myscript.py) will get the input from another script with pipe.

Assume that I call the scripts like this;

$ python writer.py | python myscript.py 

And here is the codes;

// writer.py
import time, sys

def main():
    while True:
        print "test"
        sys.stdout.flush()
        time.sleep(1)

main()

//myscript.py
def get_input():
    while True:
        text = sys.stdin.readline()
        print "hello " + text
        time.sleep(3)

if __name__ == '__main__':        
    p1 = Process(target=get_input, args=())
    p1.start()

this is clearly not working, since the sys.stdin objects are different for main process and p1. So I have tried this to solve it,

//myscript.py
def get_input(temp):
    while True:
        text = temp.readline()
        print "hello " + text
        time.sleep(3)

if __name__ == '__main__':        
    p1 = Process(target=get_input, args=(sys.stdin,))
    p1.start()

but I come across with this error;

Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "in.py", line 12, in get_input
    text = temp.readline()
ValueError: I/O operation on closed file

So, I guess that main's stdin file closed and I can't read from it. At this conjunction, how can I pass main's stdin file to another process? If passing stdin is not possible, how can I use main's stdin from another process?

update: Okay, I need to clarify my question since people think using multiprocessing is not really necessary. consider myscript.py like this;

//myscript.py
def get_input():
    while True:
        text = sys.stdin.readline()
        print "hello " + text
        time.sleep(3)

def do_more_things():
    while True:
        #// some code here
        time.sleep(60*5)

if __name__ == '__main__':        
    p1 = Process(target=get_input, args=())
    p1.start()

    do_more_things()

so, I really need to run get_input() function parallelly with main function (or other sub processes). Sorry for the conflicts, I have a decent English, and I guess I couldn't be clear on this question. I would appreciate if you guys can tell me if i can use the main processes STDIN object in another process.

thanks in advance.

Answer 1

The simplest thing is to swap get_input() and do_more_things() i.e., read sys.stdin in the parent process:

def get_input(stdin):
    for line in iter(stdin.readline, ''):
        print("hello", line, end='')
    stdin.close()

if __name__ == '__main__':
    p1 = mp.Process(target=do_more_things)
    p1.start()
    get_input(sys.stdin)

The next best thing is to use a Thread() instead of a Process() for get_input():

if __name__ == '__main__':
    t = Thread(target=get_input, args=(sys.stdin,))
    t.start()
    do_more_things()

If the above doesn't help you could try os.dup():

newstdin = os.fdopen(os.dup(sys.stdin.fileno()))
try: 
   p = Process(target=get_input, args=(newstdin,))
   p.start()    
finally:
   newstdin.close() # close in the parent
do_more_things()

Answer 2

Each new process created with the multiprocessing module gets its own PID, and therefore it's own standard input device and output devices, even if they're both writing to the same terminal, hence the need for locks.

You're already creating two processes by separating the content into two scripts, and creating a third process with get_input(). get_input could read the standard input if it was a thread instead of a process. Then, no need to have a sleep function in the reader.

## reader.py
from threading import Thread
import sys

def get_input():
    text = sys.stdin.readline()
    while len(text) != 0:
        print 'hello ' + text
        text = sys.stdin.readline()

if __name__ == '__main__':
    thread = Thread(target=get_input)
    thread.start()
    thread.join()