Python how to do multiprocessing inside of a class?

Question

I have a code structure that looks like this:

Class A:   def __init__(self):     processes = []     for i in range(1000):       p = Process(target=self.RunProcess, args=i)       processes.append[p]      # Start all processes     [x.start() for x in processes]    def RunProcess(self, i):     do something with i... 

Main script:

myA = A() 

I can't seem to get this to run. I get a runtime error "An attempt has been made to start a new process before the current process has finished its bootstrapping phase."

How do I get multiple processing working for this? If I use Threading, it works fine but it is as slow as sequential... And I'm also afraid that multiple processing will also be slow because it takes longer for the the process to be created?

Any good tips? Many thanks in advance.

Answer 1

There are a couple of syntax issues that I can see in your code:

• args in Process expects a tuple, you pass an integer, please change line 5 to:

  p = Process(target=self.RunProcess, args=(i,))

• list.append is a method and arguments passed to it should be enclosed in (), not [], please change line 6 to:

  processes.append(p)

As @qarma points out, its not good practice to start the processes in the class constructor. I would structure the code as follows (adapting your example):

import multiprocessing as mp from time import sleep  class A(object):     def __init__(self, *args, **kwargs):         # do other stuff         pass      def do_something(self, i):         sleep(0.2)         print('%s * %s = %s' % (i, i, i*i))      def run(self):         processes = []          for i in range(1000):             p = mp.Process(target=self.do_something, args=(i,))             processes.append(p)          [x.start() for x in processes]   if __name__ == '__main__':     a = A()     a.run()