Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Value update in manager.dict not reflected

I expect the following code to print [{0: 100}], since it do the plus in updateList for a hundred time In turn, it prints [{0: 0}] what's the problem and how to rectify it?

from multiprocessing import Process, Lock, Value,Manager
class myWorker:
    def __init__(self, lock, driver, i):
        self.idx=i
        self.driver=driver
        self.lock=lock

    def run(self):
        self.driver.updateList(self.lock,self.idx)

class driver(object):

    def __init__(self):
        manager=Manager()
        self.lock=Lock()
        self.lst=manager.list()
        self.dict1=manager.dict({0:0})
        self.lst.append(self.dict1)

    def workerrun(self,lock, i):
        worker1=myWorker(lock,self,i)
        worker1.run()

    def run(self):
        D=[Process(target=self.workerrun,args=(self.lock,i)) for i in range(10)]
        for d in D:
            d.start()
        for d in D:
            d.join()

    def updateList(self,l,i):
        with self.lock: # acquire lock
            for j in range(10):
                self.lst[0][0]+=1

            print ("update from", i)

if __name__=='__main__':
    dr=driver()
    dr.run()
    print(dr.lst)
like image 765
william007 Avatar asked Oct 25 '14 12:10

william007


2 Answers

 def updateList(self,l,i):
        with self.lock: # acquire lock
            for j in range(10):
                d = self.lst[0]
                d[0] += 1
                self.lst[0]=d
                print ("update from", i,self.lst[0])

from the docs

Note Modifications to mutable values or items in dict and list proxies will not be propagated through the manager, because the proxy has no way of knowing when its values or items are modified. To modify such an item, you can re-assign the modified object to the container proxy:

Seeing as you already have dict1, you can update directly:

   def updateList(self,l,i):
        with self.lock: # acquire lock
            for j in range(10):
                self.dict1[0]+=1
                print ("update from", i,self.lst[0])
like image 183
Padraic Cunningham Avatar answered Oct 17 '22 22:10

Padraic Cunningham


multiprocessing.Manager() returns a multiprocessing.managers.SyncManager if you see the when you created the list and the dictionary, you actually got their proxies. That means, what you appended to the list not the {0:0} that you created inside the manager but a proxy(copy).

self.dict1=manager.dict({0:0})
#this:
self.lst.append(self.dict1)
#is the same as doing this:
self.lst.append({0:0})   

So, in the updatelist method:

def updateList(self,l,i):
    with self.lock: # acquire lock
        for j in range(10):
            self.lst[0][0]+=1

            # is the same as doing:
            # example: self.lst == [{0:x}]
            proxy = self.lst
            # you get a copy of actual list
            # proxy == [{0:x}]
            dict = proxy[0]
            dict[0]+=1
            # proxy == [{0:x+1}]
            # BUT
            # self.lst == [{0:x}]

That means you are making copies, changing them, and then not using them. You need to change to assign the list with the new value so that it changes for all processes:

def updateList(self,l,i):
    with self.lock: # acquire lock
        dict0 = self.lst[0]
        for j in range(10):
            dict0[0]+=1
        self.lst[0] = dict0
like image 30
Rui Botelho Avatar answered Oct 17 '22 23:10

Rui Botelho