Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get return value for multi-processing functions in python

I have two functions to run in parallel and each of them returns a value. I need to wait for both functions to finish and then process the returns from them. How could I achieve this in python. Assume

def fun1():
   #do some calculation#
   return val1

def fun2():
   #do some calculation#
   return val2

I want fun1 and fun2 to run in parallel and then need to calculate

valsum = val1+val2
like image 945
TheStupidOne Avatar asked Jun 17 '16 05:06

TheStupidOne


1 Answers

Using concurrent.futures:

from concurrent.futures import ProcessPoolExecutor as Executor
#from concurrent.futures import ThreadPoolExecutor as Executor  # to use threads
with Executor() as executor:
    future1 = executor.submit(fun1, arg1, arg2, ...)
    future2 = executor.submit(fun2, arg1, arg2, ...)
    val1, val2 = future1.result(), future2.result()
    valsum = val1 + val2

concurrent.futures.Executor.submit schedules the function to be executed, and returns a concurrent.futures.Future object.

concurrent.futures.Future.resultreturns the return value returned by the function.


Using multiprocessing.pool:

from multiprocessing.pool import Pool
#from multiprocessing.pool import ThreadPool as Pool  # to use threads
with multiprocessing.pool.Pool() as pool:
    result1 = pool.apply_async(fun1, (arg1, arg2, ...))
    result2 = pool.apply_async(fun2, (arg1, arg2, ...))
    val1, val2 = result1.get(), result2.get()
    valsum = val1 + val2

multiprocessing.pool.Pool.apply_async returns an AsyncResult object, and AsyncResult.get will return the return value of the function once it finish and the result arrive.

like image 138
falsetru Avatar answered Oct 13 '22 16:10

falsetru