Closing Pipe in Python

Question

import multiprocessing as mp
import time

"""
1. send item via pipe
2. Receive on the other end by a generator
3. if the pipe is closed on the sending side, retrieve
all item left and then quit.
"""

def foo(conn):
    for i in range(7):
        time.sleep(.3)
        conn.send(i)
    conn.close()

def bar(conn):
    while True:
        try:
            yield conn.recv()
        except EOFError:
            break


if __name__ == '__main__':
    """Choose which start method is used"""
    recv_conn, send_conn = mp.Pipe(False)
    p = mp.Process(target = foo, args = (send_conn,)) # f can only send msg.
    p.start()
#    send_conn.close()
    for i in bar(recv_conn):
        print(i)

I'm using Python 3.4.1 on Ubuntu 14.04 and the code is not working. At the end of the program, there is no EOFError, which should terminates the code, although the Pipe has been closed. Closing the Pipe inside a function does not close the Pipe. Why is this the case?

Answer 1

Uncomment your send_conn.close() line. You should be closing pipe ends in processes that don't need them. The issue is that once you launch the subprocess, the kernel is tracking two open references to the send connection of the pipe: one in the parent process, one in your subprocess.

The send connection object is only being closed in your subprocess, leaving it open in the parent process, so your conn.recv() call won't raise EOFError. The pipe is still open.

This answer may be useful to you as well.

I verified that this code works in Python 2.7.6 if you uncomment the send_conn.close() call.