I am creating a program like this.
from __future__ import print_function
import multiprocessing
dict1 = dict( (k,v) for (k,v) in zip(range(0,11),range(50,61)))
dict2={}
dict3={}
def fun1(dct):
#How can I process `dct` as `multiprocessing.Pool` would act like a loop sending chunks of iterable?
#I can do:
# for i in dct:
# dict2.update({dct[i]:i})
# but `multiprocessing.Pool` will do the looping part automatically. In this case what should be done to index `dct`?
#dict2.update({dct[i]:i})
return dict2
if __name__ == '__main__':
p=multiprocessing.Pool(2)
dict3=p.map(fun1,dict1)
p.close()
p.join()
print(dict3) #write in file
I want to modify global variable dict2 inside function fun1 and return the updated global variable to main function to print (write it to a file). However, before that, I am getting error TypeError: 'int' object is unsubscriptable. How can I get this to work?
I read some questions on how to modify a global variable using keyword global, but placing global dict2 in fun1() will reset the variable every time.
Update:
How can I process dct like dct[0] as multiprocessing.Pool would act like a loop sending chunks of iterable?
This gets you your final version of dict2, but I'm not sure that multiprocessing buys you anything in terms of performance.
Warning: Python3 alert
import multiprocessing
dict1 = dict( (k,v) for (k,v) in zip(range(0,11),range(50,61)))
dict2={}
def fun1(dct):
key, value, dict2 = dct #unpack tuple
# This code would not update dict2
# dict2.update({value:key})
# return the value and key reversed. I assume this is what you are after.
return {value:key}
if __name__ == '__main__':
p=multiprocessing.Pool(2)
# pass the tuple of (key,value,dict2) into each call of fun1
dict3=p.map(fun1,((key,value,dict2) for key, value in dict1.items()))
p.close()
p.join()
print(dict1) # original dict
print(dict2) # remains empty
print(dict3) # this is a list of the results
for d in dict3:
dict2.update(d)
# Now dict2 is populated
print(dict2)
Output:
{0: 50, 1: 51, 2: 52, 3: 53, 4: 54, 5: 55, 6: 56, 7: 57, 8: 58, 9: 59, 10: 60}
{}
[{50: 0}, {51: 1}, {52: 2}, {53: 3}, {54: 4}, {55: 5}, {56: 6}, {57: 7}, {58: 8}, {59: 9}, {60: 10}]
{50: 0, 51: 1, 52: 2, 53: 3, 54: 4, 55: 5, 56: 6, 57: 7, 58: 8, 59: 9, 60: 10}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With