Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multiprocessing manager class object thread/process safe

I have the following class that is shared between multiple consumers (using producer/consumer methodology). My question involves the methods called on this class. Do I need to implement locks or is the manager class thread safe?

import multiprocessing as mp
from multiprocessing.manager import BaseManager

class SampleClass(object):

    def __init__(self):
        self._count = 0

    # Does locking need to be implemented here?
    def increment(self):
        self._count += 1

BaseManager.register('SampleClass', SampleClass)
manager = BaseManager()
manager.start()

instance = manager.SampleClass()

jobs = []
for i in range(0, 5):
    p = mp.Process(target=some_func, args=(instance,))
    jobs.append(p)
    p.start()

for p in jobs:
    p.join()
like image 951
Aaron Phalen Avatar asked Jul 05 '26 05:07

Aaron Phalen


1 Answers

Prior answers are incomplete as they don't address thread safety. Poster asked about threads.
SyncManager section of docs don't mention Queues and threads. Without the code for some_func we don't know whether child processes spawn multiple threads.

It appears that multiprocessing queues are both thread and process safe. According to this section of Python multiprocessing docs:

Queues are thread and process safe.

That's for multiprocessing.Queue not multiprocessing.manager.Queue or SyncManager.Queue. However answers to this question indicate that manager.Queue is just a multiprocessing.Queue run by a manager. So manager.Queue and SyncManager.Queue are thread safe too.

like image 91
Ed_ Avatar answered Jul 08 '26 00:07

Ed_