Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Assigning return value of function to a variable, with multiprocessing? And a problem about IDLE?

I'm trying to understand multiprocessing in python.

from multiprocessing import Process

def multiply(a,b):
    print(a*b)
    return a*b

if __name__ == '__main__':
    p = Process(target= multiply, args= (5,4))
    p.start()
    p.join()
    print("ok.")

In this codeblock, for example, if there was an variable that called "result". How can we assign return value of multiply function to "result"?

And a little problem about IDLE: when i'm tried to run this sample with Python Shell, it doesn't work properly? If i double click .py file, output is like that:

20
ok.

But if i try to run this in IDLE:

ok.

Thanks...

like image 993
sleepnir Avatar asked Aug 25 '11 17:08

sleepnir


People also ask

What is multiprocessing dummy?

dummy module module provides a wrapper for the multiprocessing module, except implemented using thread-based concurrency. It provides a drop-in replacement for multiprocessing, allowing a program that uses the multiprocessing API to switch to threads with a single change to import statements.

What is the multiprocessing function in Python?

multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads.

How does multiprocessing lock work in Python?

Python provides a mutual exclusion lock for use with processes via the multiprocessing. Lock class. An instance of the lock can be created and then acquired by processes before accessing a critical section, and released after the critical section. Only one process can have the lock at any time.


2 Answers

Ok, i somehow managed this. I looked to python documentation, and i learnt that: with using Queue class, we can get return values from a function. And final version of my code is like this:

from multiprocessing import Process, Queue

def multiply(a,b,que): #add a argument to function for assigning a queue
    que.put(a*b) #we're putting return value into queue

if __name__ == '__main__':
    queue1 = Queue() #create a queue object
    p = Process(target= multiply, args= (5,4,queue1)) #we're setting 3rd argument to queue1
    p.start()
    print(queue1.get()) #and we're getting return value: 20
    p.join()
    print("ok.")

And there is also a pipe() function, i think we can use pipe() function,too. But Queue worked for me, now.

like image 149
sleepnir Avatar answered Oct 01 '22 18:10

sleepnir


Does this help? This takes a list of functions (and their arguments), runs them in parallel, and returns their outputs.: (This is old. Much newer version of this is at https://gitlab.com/cpbl/cpblUtilities/blob/master/parallel.py )

def  runFunctionsInParallel(listOf_FuncAndArgLists):
    """
    Take a list of lists like [function, arg1, arg2, ...]. Run those functions in parallel, wait for them all to finish, and return the list of their return values, in order.

(This still needs error handling ie to ensure everything returned okay.)

    """
    from multiprocessing import Process, Queue

    def storeOutputFFF(fff,theArgs,que): #add a argument to function for assigning a queue
        print 'MULTIPROCESSING: Launching %s in parallel '%fff.func_name
        que.put(fff(*theArgs)) #we're putting return value into queue

    queues=[Queue() for fff in listOf_FuncAndArgLists] #create a queue object for each function
    jobs = [Process(target=storeOutputFFF,args=[funcArgs[0],funcArgs[1:],queues[iii]]) for iii,funcArgs in enumerate(listOf_FuncAndArgLists)]
    for job in jobs: job.start() # Launch them all
    for job in jobs: job.join() # Wait for them all to finish
    # And now, collect all the outputs:
    return([queue.get() for queue in queues])
like image 36
CPBL Avatar answered Oct 01 '22 19:10

CPBL