multiprocessing Listeners and Clients between python and pypy

Question

Is it possible to have a Listener server process and a Client process where one of them uses a python interpreter and the other a pypy interpreter?

Would conn.send() and conn.recv() interoperate well?

Answer 1

I tried it out to see:

import sys
from multiprocessing.connection import Listener, Client

address = ('localhost', 6000)

def client():
    conn = Client(address, authkey='secret password')
    print conn.recv_bytes()
    conn.close()

def server():
    listener = Listener(address, authkey='secret password')
    conn = listener.accept()
    print 'connection accepted from', listener.last_accepted
    conn.send_bytes('hello')
    conn.close()
    listener.close()

if __name__ == '__main__':
    if sys.argv[1] == 'client':
        client()
    else:
        server()

Here are the results I got:

• CPython 2.7 + CPython 2.7: working
• PyPy 1.7 + PyPy 1.7: working
• CPython 2.7 + PyPy 1.7: not working
• CPython 2.7 + PyPy Nightly (pypy-c-jit-50911-94e9969b5f00-linux64): working

When using PyPy 1.7 (doesn't matter which is the server and which is the client), an error is reported with IOError: bad message length. This also mirrors the report on the pypy-dev mailing list. However, this was recently fixed (it works in nightly build), so the next version (presumably 1.8) should have it fixed as well.

In general, this works because the multiprocessing module uses Python's pickle module, which is stable and supported across multiple Python implementations, even PyPy.