Create child processes inside a child process with Python multiprocessing failed

Question

I observed this behavior when trying to create nested child processes in Python. Here is the parent program parent_process.py:

import multiprocessing
import child_process

pool = multiprocessing.Pool(processes=4)
for i in range(4):
        pool.apply_async(child_process.run, ())
pool.close()
pool.join()

The parent program calls the "run" function in the following child program child_process.py:

import multiprocessing

def run():
        pool = multiprocessing.Pool(processes=4)
        print 'TEST!'
        pool.close()
        pool.join()

When I run the parent program, nothing was printed out and the program exited quickly. However, if print 'TEST!' is moved one line above (before the nested child processes are created), 'TEST!' are printed for 4 times.

Because errors in a child process won't print to screen, this seems to show that the program crashes when a child process creates its own nested child processes.

Could anyone explain what happens behind the scene? Thanks!

Answer 1

According to the documentation for multiprocessing, daemonic processes cannot spawn child processes.

multiprocessing.Pool uses daemonic processes to ensure that they don't leak when your program exits.