Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to inherit "Manager().list" in python?

I've implemented my own "DataSet" class which inherited from list and I need to port that code to run in multiprocessing mode. Here is a sample of my class:

class DataSet(list):
    @property
    def n_training(self):
        return len(self) * 0.8
    @property
    def n_validation(self):
        return len(self) * 0.2

Beside creating an instance of "Manager().list" in this class, are there any more simple way to do this like:

from multiprocessing import Manager
class DataSet(Manager().list):

Please note that "Manager().list" is the representation of

manager = Manager()
manager.list()
like image 395
Jessada Thutkawkorapin Avatar asked May 22 '26 10:05

Jessada Thutkawkorapin


1 Answers

According to the suggestion from @senderle and @Martijn Pieters, I managed to get a solution

from multiprocessing.managers import BaseManager
from multiprocessing.managers import BaseProxy

class DataSetClass(list):
    @property
    def n_training(self):
        return len(self) * 0.8
    @property
    def n_validation(self):
        return len(self) * 0.2

class DataSetProxy(BaseProxy):
    _exposed_ = ('append', '__len__')
    def append(self, item):
        return self._callmethod('append', item)
    def __len__(self):
        return self._callmethod('__len__')

class MyManager(BaseManager):
    pass

MyManager.register('DataSet', DatasetClass, DataSetProxy)

if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    dataset = manager.DataSet()
    dataset.append('item1')
    dataset.append('item2')
    print len(dataset)
like image 84
Jessada Thutkawkorapin Avatar answered May 23 '26 22:05

Jessada Thutkawkorapin



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!