Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the result of multiprocessing.Pool.apply_async

I want to get the result of the function run by Pool.apply_async in Python.

How to assign the result to a variable in the parent process? I tried to use callback but it seems complicated.

like image 755
THN Avatar asked Mar 16 '17 19:03

THN


People also ask

What does Apply_async return?

The apply_async() function returns an AsyncResult, whereas the apply() function returns the result of the target function. The apply_async() function can execute callback functions when the task is complete, whereas the apply() function cannot execute callback functions.

How does a pool work in multiprocessing?

Pool allows multiple jobs per process, which may make it easier to parallel your program. If you have a numbers jobs to run in parallel, you can make a Pool with number of processes the same number of as CPU cores and after that pass the list of the numbers jobs to pool. map.

How do you close a multiprocessing pool?

The process pool can be shutdown by calling the Pool. close() function. This will prevent the pool from accepting new tasks. Once all issued tasks are completed, the resources of the process pool, such as the child worker processes, will be released.

What is multiprocessing Freeze_support?

multiprocessing. freeze_support() This function will allow a frozen program to create and start new processes via the multiprocessing. Process class when the program is frozen for distribution on Windows. If the function is called and the program is not frozen for distribution, then it has no effect.

How do you get multiprocessing in Python?

In this example, at first we import the Process class then initiate Process object with the display() function. Then process is started with start() method and then complete the process with the join() method. We can also pass arguments to the function using args keyword.


2 Answers

The solution is very simple:

import multiprocessing

def func():
    return 2**3**4

p = multiprocessing.Pool()
result = p.apply_async(func).get()
print(result)

Since Pool.apply_async() returns an AsyncResult, you can simply get the result from the AsyncResult.get() method.

Hope this helps!

like image 196
linusg Avatar answered Sep 29 '22 12:09

linusg


Well an easy way would be to have a helper class like this:

class Result():
    def __init__(self):
        self.val = None

    def update_result(self, val):
        self.val = val

result = Result()

def f(x):
    return x*x

pool.apply_async(f, (10,), callback=result.update_result)

When the thread runs and calculates the result it will call your callback which will update the result.val.

In any case you need to check that the thread has finished.

like image 43
Giannis Spiliopoulos Avatar answered Sep 29 '22 12:09

Giannis Spiliopoulos