The docs (python 3.4) explain that with spawn
, "the child process will only inherit those resources necessary to run the process object's run()
method".
But which objects are "necessary"? The way I read it suggested to me that all the objects that can be reached from inside run()
are "necessary", including arguments passed as args
to Process.__init__
, plus whatever is stored in global variables, as well as classes, functions defined in global scope and their attributes. However, this is incorrect; the following code confirms that the objects stored in global variables aren't inherited:
# running under python 3.4 / Windows
# but behaves the same under Unix
import multiprocessing as mp
x = 0
class A:
y = 0
def f():
print(x) # 0
print(A.y) # 0
def g(x, A):
print(x) # 1
print(A.y) # 0; really, not even args are inherited?
def main():
global x
x = 1
A.y = 1
p = mp.Process(target = f)
p.start()
q = mp.Process(target = g, args = (x, A))
q.start()
if __name__=="__main__":
mp.set_start_method('spawn')
main()
Is there a clear rule that states which objects are inherited?
EDIT:
To confirm: running this on Ubuntu produces the same output. (Thanks to @mata for clarifying that I forgot add global x
to main()
. This omission made my example confusing; it would also affect the result if I were to switch 'spawn'
to 'fork'
under Ubuntu. I now added global x
to the code above.)
In multiprocessing , processes are spawned by creating a Process object and then calling its start() method.
shared_memory — Shared memory for direct access across processes. New in version 3.8. This module provides a class, SharedMemory , for the allocation and management of shared memory to be accessed by one or more processes on a multicore or symmetric multiprocessor (SMP) machine.
A thread is a sequence of instructions that are being executed within the context of a process. One process can spawn multiple threads but all of them will be sharing the same memory.
The term "spawn" means the creation of a process by a parent process. The parent process can of course continue its execution asynchronously or wait until the child process ends its execution. The multiprocessing library of Python allows the spawning of a process through the following steps: Build the object process.
This has to do with the way classes are pickled when being sent to the spawned Process. The pickled version of a class doesn't really contain its internal state, but only the module and the name of the class:
class A:
y = 0
pickle.dumps(A)
# b'\x80\x03c__main__\nA\nq\x00.'
There is no information about y
here, it's comparable to a reference to the class.
The class will be unpickled in the spawned process when passed as argumeht to g
, which will import its module (here __main__
) if neccessary and return a reference to the class, therefore changes made to it in your main
function won't affect it as the if __name__ == "__main__"
block won't be executed in the subprocess. f
directly uses the class in its module, so the effect is basically the same.
The reason why x
shows different values is a little different. Your f
function will print the global variable x
from the module. In your main()
function you have another local variable x
, so setting x = 1
here won't affect the module level x
in neither processes. It's passed to g
as argument, so in this case it will alays have the local value of 1.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With