How to share data between Python processes?

Question

I'm using multiprocessing to create a sub-process to my Python app. I would like to share data between my parent process and the child process. it's important to mention that I need to share this asynchronously, means that the child process and the parent process will update the data during the code running.

What would be the best way to perform that?

Answer 1

Here's an example of multiprocess-multithread and sharing a couple variables:

from multiprocessing import Process, Queue, Value, Manager
from ctypes import c_bool
from threading import Thread


ps = []

def yourFunc(pause, budget):
    while True:
        print(budget.value, pause.value)
        ##set value
        pause.value = True  
        ....

def multiProcess(threads, pause, budget):
    for _ in range(threads):
        t = Thread(target=yourFunc(), args=(pause, budget,)) 
        t.start()
        ts.append(t)
        time.sleep(3)

if __name__ == '__main__':
    pause = Value(c_bool, False)
    budget = Value('i', 5000)

    for i in range(2):
        p = Process(target=multiProcess, args=(2, pause, budget))
        p.start()
        ps.append(p) 

Answer 2

This is one simple example from python documentation -

from multiprocessing import Process, Queue

def f(q):
    q.put([42, None, 'hello'])

if __name__ == '__main__':
    q = Queue()
    p = Process(target=f, args=(q,))
    p.start()
    print q.get()    # prints "[42, None, 'hello']"
    p.join()

You can use pipe as well, Refer for more details - https://docs.python.org/2/library/multiprocessing.html