Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficient Python to Python IPC [closed]

What would be an inter-process communication (IPC) framework\technique with the following requirements:

  • Transfer native Python objects between two Python processes
  • Efficient in time and CPU (RAM efficiency irrelevant)
  • Cross-platform Win\Linux
  • Nice to have: works with PyPy

UPDATE 1: the processes are on the same host and use the same versions of Python and other modules

UPDATE 2: the processes are run independently by the user, no one of them spawns the others

like image 733
Jonathan Livni Avatar asked Oct 20 '11 17:10

Jonathan Livni


People also ask

How do you communicate between two processes in Python?

Passing Messages to Processes A simple way to communicate between process with multiprocessing is to use a Queue to pass messages back and forth. Any pickle-able object can pass through a Queue. This short example only passes a single message to a single worker, then the main process waits for the worker to finish.

What is IPC in Python?

Interprocess Communication and Networking — Python 3.6.

Which Python module provides support for networking and inter process communication?

asyncio — Asynchronous I/O.


1 Answers

Native objects don't get shared between processes (due to reference counting).

Instead, you can pickle them and share them using unix domain sockets, mmap, zeromq, or an intermediary such a sqlite3 that is designed for concurrent accesses.

like image 144
Raymond Hettinger Avatar answered Sep 19 '22 02:09

Raymond Hettinger