Python multiprocessing pool never finishes

Question

I am running the following (example) code:

from multiprocessing import Pool

def f(x):
  return x*x

pool = Pool(processes=4)
print pool.map(f, range(10))

However, the code never finishes. What am I doing wrong?

The line

pool = Pool(processes=4)

completes successfully, it appears to stop in the last line. Not even pressing ctrl+c interrupts the execution. I am running the code inside an ipython console in Spyder.

Answer 1

from multiprocessing import Pool


def f(x):
    return x * x


def main():
    pool = Pool(processes=3)  # set the processes max number 3
    result = pool.map(f, range(10))
    pool.close()
    pool.join()
    print(result)
    print('end')


if __name__ == "__main__":
    main()

The key step is to call pool.close() and pool.join() after the processes finished. Otherwise the pool is not released. Besides, you should create the pool in the main process by putting the codes within if __name__ == "__main__":

Answer 2

Your constructor is throwing the interpreter off into a thread producing factory for some reason. You first need to stop all the threads are now running and there will be tons. If you bring up the task manager you will see tons of rogue python.exe tasks. To kill them in bulk try:

taskkill /F /IM python.exe

You would need to do the above a couple of times and make sure the task manager does not show anymore python.exe tasks. This will also kill you spyder instance. So make sure you save.

Now change your code to the following:

from multiprocessing import Pool

def f(x):
  return x*x

if (__name__ == '__main__'):
   pool = Pool(4)
   print pool.map(f, range(10))

Note that I have removed the processes named argument.