I've implemented my own "DataSet" class which inherited from list and I need to port that code to run in multiprocessing mode. Here is a sample of my class:
class DataSet(list):
@property
def n_training(self):
return len(self) * 0.8
@property
def n_validation(self):
return len(self) * 0.2
Beside creating an instance of "Manager().list" in this class, are there any more simple way to do this like:
from multiprocessing import Manager
class DataSet(Manager().list):
Please note that "Manager().list" is the representation of
manager = Manager()
manager.list()
According to the suggestion from @senderle and @Martijn Pieters, I managed to get a solution
from multiprocessing.managers import BaseManager
from multiprocessing.managers import BaseProxy
class DataSetClass(list):
@property
def n_training(self):
return len(self) * 0.8
@property
def n_validation(self):
return len(self) * 0.2
class DataSetProxy(BaseProxy):
_exposed_ = ('append', '__len__')
def append(self, item):
return self._callmethod('append', item)
def __len__(self):
return self._callmethod('__len__')
class MyManager(BaseManager):
pass
MyManager.register('DataSet', DatasetClass, DataSetProxy)
if __name__ == '__main__':
manager = MyManager()
manager.start()
dataset = manager.DataSet()
dataset.append('item1')
dataset.append('item2')
print len(dataset)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With