How to print out the process name of a process from python's multiprocessing pool

Question

I'm trying to figure out what's happening in each process by printing out the process name, identifier, anything.

Below is some test code (code credit to this SO question:)

def f(x):
   print multiprocessing.current_process().name
   return x * x

def b():
   p = multiprocessing.Pool()
   print p.map(f, range(6))

def main():
   b()

The output it produces is:

PoolWorker-6
PoolWorker-10
PoolWorker-10
PoolWorker-11
PoolWorker-14
PoolWorker-15
[0, 1, 4, 9, 16, 25]

Which is what I think I want/need to help me debug what's wrong with my program. However, my code is slightly different (legacy code I need to fix from someone else). The code I have below doesn't print the nice PoolWorker ID.

def f(x):
   print multiprocessing.current_process().name
   return x * x

def b():
   p = multiprocessing.Pool()
   for i in range(10):
      p.apply_async(f(i))

def main():
   b()

When I run this code, it prints out:

MainProcess
MainProcess
MainProcess
MainProcess
MainProcess
MainProcess

So my code uses Pool.apply_sync() instead of Pool.map(). How do I print out some unique identifier/name for each process so I have insight into what's happening? Thanks in advance for your help.

Answer 1

Note - Be careful while opening too many parallel process. apply_sync is useful but you have to set max limit of parallel process you want to open. By default it's None. It is also important to note that if any pool worker would be free then that would be used. So put some sleep in f(x) to make sure none of them should be free until you create all parallel process.

import multiprocessing
import time
def f(x):
   print multiprocessing.current_process().name
   time.sleep(4)
   return x * x

def b():
   p = multiprocessing.Pool(processes=4)
   for i in range(4):
      p.apply_async(f, args=(i,))
   p.close()
   p.join()
b()

PoolWorker-1
PoolWorker-3
PoolWorker-2
PoolWorker-4

Answer 2

Try that :

def f(x):
   print multiprocessing.current_process().name
   return x * x

def b():
   p = multiprocessing.Pool()
   for i in range(10):
      p.apply_async(f, args=(i,))
   p.close()
   p.join()


def main():
   b()