Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why python multiprocessing module cause CPU completely run out? [duplicate]

Tags:

python

Possible Duplicate:
Multiprocessing launching too many instances of Python VM

I am trying python 2.6 multiprocessing module with this simple code snippet.

from multiprocessing import Pool
p = Pool(5)
def f(x):
    return x*x

print p.map(f, [1,2,3])

But this code cause my OS stopped responding. It looks like the CPU is too busy. What's wrong with my code?

BTW : it seems that multiprocessing module is a bit dangerous. I had to restart my computer.

like image 770
ablmf Avatar asked Jan 20 '10 15:01

ablmf


People also ask

Does multiprocessing require multiple cores?

Common research programming languages use only one processor The “multi” in multiprocessing refers to the multiple cores in a computer's central processing unit (CPU). Computers originally had only one CPU core or processor, which is the unit that makes all our mathematical calculations possible.

How many processes can acquire an instance of multiprocessing lock concurrently?

Only one process can acquire the lock at a time, and then once acquired, blocks and then reenters the same lock again to report the done message via the report() function.

Is multiprocessing concurrent?

Both multithreading and multiprocessing allow Python code to run concurrently. Only multiprocessing will allow your code to be truly parallel. However, if your code is IO-heavy (like HTTP requests), then multithreading will still probably speed up your code.

How do I know if multiprocessing is working?

We can check if a process is alive via the multiprocessing. Process. is_alive() method.


1 Answers

You aren't protecting the entry point at all, so each subprocess is trying to start the same map call and so on (into infinity!). Try the following:

if __name__ == "__main__":
    print p.map(f, [1,2,3])

See this section of the module's documentation.

like image 115
jkp Avatar answered Oct 18 '22 06:10

jkp