Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python multiprocessing arguments: deep copy?

from multiprocessing import Process
# c is a container
p = Process(target = f, args = (c,))
p.start()

I assume a deep copy of c is passed to function f because shallow copy would make no sense in the case of a new process (the new process doesn't have access to the data from the calling process).

But how is this deep copy defined? There is a whole set of notes in the copy.deepcopy() documentation, do all these notes apply here as well? The multiprocessing documentation says nothing...

like image 301
max Avatar asked May 12 '11 19:05

max


1 Answers

When you create a Process instance, under the hood Python issues a fork(). This creates a child process whose memory space is an exact copy of its parent -- so everything existing at the time of the fork is copied.

On Linux this is made efficient through "copy-on-write". From the fork man page:

fork() creates a child process that differs from the parent process only in its PID and PPID, and in the fact that resource utilizations are set to 0. File locks and pending signals are not inherited.

Under Linux, fork() is implemented using copy-on-write pages, so the only penalty that it incurs is the time and memory required to duplicate the parent's page tables, and to create a unique task structure for the child.

like image 102
samplebias Avatar answered Oct 04 '22 20:10

samplebias