What is the meaning of `context` argument in `multiprocessing.pool.Pool`?

Question

context is an optional argument in the constructor of class multiprocessing.pool.Pool. Documentation only says:

context can be used to specify the context used for starting the worker processes. Usually a pool is created using the function multiprocessing.Pool() or the Pool() method of a context object. In both cases context is set appropriately.

It doesn't clarify what a "context object" is, why class Pool constructor needs it, and what it means that it "is set appropriately" in the mentioned scenarios.

Answer 1

Depending on the platform, multiprocessing supports three ways to start a process. These start methods are:

• spawn:

  The parent process starts a fresh python interpreter process.
  Available on Unix and Windows. The default on Windows.

• fork:

  The parent process uses os.fork() to fork the Python interpreter. Available on Unix only. The default on Unix.

• forkserver

  When the program starts and selects the forkserver start method, a server process is started. From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process. The fork server process is single threaded so it is safe for it to use os.fork(). No unnecessary resources are inherited.

  Available on Unix platforms which support passing file descriptors over Unix pipes.

---

To select a start method you use the set_start_method() in the if __name__ == '__main__' clause of the main module. For example:

import multiprocessing as mp  def foo(q):     q.put('hello')  if __name__ == '__main__':     mp.set_start_method('spawn')     q = mp.Queue()     p = mp.Process(target=foo, args=(q,))     p.start()     print(q.get())     p.join() 

Alternatively, you can use get_context() to obtain a context object. Context objects have the same API as the multiprocessing module, and allow one to use multiple start methods in the same program.

import multiprocessing as mp  def foo(q):     q.put('hello')  if __name__ == '__main__':     ctx = mp.get_context('spawn')     q = ctx.Queue()     p = ctx.Process(target=foo, args=(q,))     p.start()     print(q.get())     p.join() 

This is where the context object came from!