Is there a way to check if a module is being loaded by multiprocessing standard module in Windows?

Question

I believe on Windows, because there is no fork, the multiprocessing module reloads modules in new Python's processes.

You are required to have this code in your main script, otherwise very nasty crashes occur

if __name__ == '__main__':
    from multiprocessing import freeze_support
    freeze_support()

I have a bunch of modules which have debug print statements in them at the module level. Therefore, the print statements get called whenever a module is being loaded.

Whenever I run something in parallel all of these print statements are executed.

My question is if there is a way to see if a module is being imported by the multiprocessing module, and if so silence those print statements?

I'm basically looking if there is something like:

 import multiprocessing
 if not multiprocessing.in_parallel_process:
     print('Loaded module: ' + __name___)

I've been unable to find it so far. Is this possible?

Answer 1

For Python 3.8 and newer use multiprocessing.parent_process() which returns None only for the main process and for earlier versions check if name attribute's value equals MainProcess.

from multiprocessing import Process, current_process

if current_process().name == 'MainProcess':
    print('Hello from the main process')
else:
    print('Hello from child process')

def f(name):
    print('hello', name)

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()

Output:

Hello from the main process
Hello from child process
hello bob

Answer 2

Yes, you can obtain information about the current process from the instance returned by multiprocessing.current_process(). In particular the Process constructor has a name argument that can be used to distinguish between child processes.

Note that in python2, if you do not specify name explicitly then the module doesn't give any guarantee on the format used, hence you cannot reliably distinguish between subprocesses and the main process: you must always explicitly specify it.

In python3 child processes are guaranteed to have a name with the format of Process-N with N a positive integer. Note that there is no guarantee on the name of the parent process, hence doing:

process.name == 'MainProcess'

Is not reliable. You should do:

import re
re.match(r'Process-\d+', process.name)