I'm trying to understand multiprocessing in python.
from multiprocessing import Process
def multiply(a,b):
print(a*b)
return a*b
if __name__ == '__main__':
p = Process(target= multiply, args= (5,4))
p.start()
p.join()
print("ok.")
In this codeblock, for example, if there was an variable that called "result". How can we assign return value of multiply function to "result"?
And a little problem about IDLE: when i'm tried to run this sample with Python Shell, it doesn't work properly? If i double click .py file, output is like that:
20
ok.
But if i try to run this in IDLE:
ok.
Thanks...
dummy module module provides a wrapper for the multiprocessing module, except implemented using thread-based concurrency. It provides a drop-in replacement for multiprocessing, allowing a program that uses the multiprocessing API to switch to threads with a single change to import statements.
multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads.
Python provides a mutual exclusion lock for use with processes via the multiprocessing. Lock class. An instance of the lock can be created and then acquired by processes before accessing a critical section, and released after the critical section. Only one process can have the lock at any time.
Ok, i somehow managed this. I looked to python documentation, and i learnt that: with using Queue
class, we can get return values from a function. And final version of my code is like this:
from multiprocessing import Process, Queue
def multiply(a,b,que): #add a argument to function for assigning a queue
que.put(a*b) #we're putting return value into queue
if __name__ == '__main__':
queue1 = Queue() #create a queue object
p = Process(target= multiply, args= (5,4,queue1)) #we're setting 3rd argument to queue1
p.start()
print(queue1.get()) #and we're getting return value: 20
p.join()
print("ok.")
And there is also a pipe()
function, i think we can use pipe()
function,too. But Queue
worked for me, now.
Does this help? This takes a list of functions (and their arguments), runs them in parallel, and returns their outputs.: (This is old. Much newer version of this is at https://gitlab.com/cpbl/cpblUtilities/blob/master/parallel.py )
def runFunctionsInParallel(listOf_FuncAndArgLists):
"""
Take a list of lists like [function, arg1, arg2, ...]. Run those functions in parallel, wait for them all to finish, and return the list of their return values, in order.
(This still needs error handling ie to ensure everything returned okay.)
"""
from multiprocessing import Process, Queue
def storeOutputFFF(fff,theArgs,que): #add a argument to function for assigning a queue
print 'MULTIPROCESSING: Launching %s in parallel '%fff.func_name
que.put(fff(*theArgs)) #we're putting return value into queue
queues=[Queue() for fff in listOf_FuncAndArgLists] #create a queue object for each function
jobs = [Process(target=storeOutputFFF,args=[funcArgs[0],funcArgs[1:],queues[iii]]) for iii,funcArgs in enumerate(listOf_FuncAndArgLists)]
for job in jobs: job.start() # Launch them all
for job in jobs: job.join() # Wait for them all to finish
# And now, collect all the outputs:
return([queue.get() for queue in queues])
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With