Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get result from Pool.starmap_async()?

I have program which computes the index of array*value and returns a string. I use .starmap_async() because I must pass two arguments to my async function. The program looks as follows:

import multiprocessing as mp
from multiprocessing import freeze_support

def go_async(self, index, value) :
        return str(index * int(value))

def log_result(self, result):
        print("Succesfully get callback! With result: ", result)

def main() :
    array = [1,3,4,5,6,7]
    pool = mp.Pool() 
    res = pool.starmap_async(go_async, enumerate(array), callback = log_result)        
    print("Final result: ", res)
    pool.close()
    pool.join()

if __name__ == '__main__':    
    freeze_support()
    main()

I would like to get a result as an array of str:

res = ['0', '3', '8', '15', '24', '35']

but I have only result:

Final result: multiprocessing.pool.MapResult object at 0x000001F7C10E51D0

How to correctly get value from .starmap_async()? Moreover, callback does not raise.

like image 310
ElConrado Avatar asked Jun 05 '19 06:06

ElConrado


People also ask

How do you pass multiple arguments in multiprocessing Python?

Use Pool. The multiprocessing pool starmap() function will call the target function with multiple arguments. As such it can be used instead of the map() function. This is probably the preferred approach for executing a target function in the multiprocessing pool that takes multiple arguments.

What is pool in multiprocessing Python?

Python multiprocessing PoolThe pool's map method chops the given iterable into a number of chunks which it submits to the process pool as separate tasks. The pool's map is a parallel equivalent of the built-in map method. The map blocks the main execution until all computations finish.

How do you get multiprocessing in Python?

In this example, at first we import the Process class then initiate Process object with the display() function. Then process is started with start() method and then complete the process with the join() method. We can also pass arguments to the function using args keyword.


1 Answers

Pool's async-methods return objects which are conceptually "explicit futures", you need to call .get() to await and receive the actual result. So res.get() will give you your result. You also need to remove self from your functions, since you are not passing an instance along your starmap-call. This currently leads to an exception within your target function and that's also the reason why your callback doesn't fire.

like image 158
Darkonaut Avatar answered Sep 22 '22 14:09

Darkonaut