Simple toy example using multiprocessing module crashes computer

Question

Trying the very simple following example causes my computer to grind to a halt, so that I have to restart. Checking task manager shows hundreds of "python.exe" tasks:

import math
from multiprocessing import Pool

pool = Pool(processes=2)
print pool.map(math.sqrt, [1,4,9,16])

I am using a dual core cpu (i5 2467m) so I thought the above would be fine.

I tried setting processes=1, which causes a slightly different problem: the task never completes but it does not cause my computer to freeze up.

Any ideas?

Answer 1

I had the same problem the first time I played around with multiprocessing. Wrap the pool generation code in a if __name__ == '__main__' block.

import math
from multiprocessing import Pool

if __name__ == '__main__':
    pool = Pool(processes=2)
    print pool.map(math.sqrt, [1,4,9,16])

What's happening is that when the subprocess is created, your module is being unpickled and rerunning the pool generation code in an infinite recursion. With the if block, the spawning code will only run in the parent instance of the module.