I currently have this piece of code (feel free to comment on it too :) )
def threaded_convert_to_png(self):
paths = self.get_pages()
pool = Pool()
result = pool.map(convert_to_png, paths)
self.image_path = result
On an Intel i7 it spawns eight workers when running on Linux; however, when running Windows 8.1 Pro it only spawns one worker. I checked and cpu_count()
returns 8 on both Linux and Windows.
P.S. This is in Python 2.7.6
There is one easy way to determine what is happends in your pool - to turn on multiprocessing debug. You can do it like this:
import logging
from multiprocessing import util
util.log_to_stderr(level=logging.DEBUG)
And on script running you will get full info about processes running, spawning and exiting.
But any way, process pool always spawn N processes (where is N - "processes" argument value or cpu_count), but tasks distribution between processes can be uneven - it depends on task run time.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With