Given the following Python code:
import multiprocessing
def unique(somelist):
return len(set(somelist)) == len(somelist)
if __name__ == '__main__':
somelist = [[1,2,3,4,5,6,7,8,9,10,11,12,13,2], [1,2,3,4,5], [1,2,3,4,5,6,7,8,9,1], [0,1,5,1]]
pool = multiprocessing.Pool()
reslist = pool.map(unique, somelist)
pool.close()
pool.join()
print "Done!"
print reslist
Now imagine, that the lists with integers in this toy example are extremely long, and what I'd like to achieve here is the following: if one of the lists in somelist returns True, kill all running processes.
This leads to two questions (and probably more which I haven't come up with):
How can I "read" or "listen" from a finished process the result, while other processes are running? If e.g. a process is dealing with [1,2,3,4,5]
from somelist
, and is finished before all other processes, how can I read out the result from that process in this very moment?
Given the case that it is possible to "read" out the result of a finished process while other are running: how can I use this result as a condition to terminate all other running processes?
e.g. If one process has finished and returned True
, how I can use this as a condition to terminate all other (still) running processes?
Use pool.imap_unordered
to view the results in any order they come up.
reslist = pool.imap_unordered(unique, somelist)
pool.close()
for res in reslist:
if res: # or set other condition here
pool.terminate()
break
pool.join()
You can iterate over an imap
reslist in your main process while the pool processes are still generating results.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With