Menu
Please explain to me what is the difference between these two classes?
I noticed multiprocessing module existed in Python 2. But functionally?
407
This provides us with a simple model for parallel execution on a multi-core machine. While concurrent futures provide a simpler interface, it is slower and less flexible when compared with using multiprocessing for parallel execution.
The ProcessPoolExecutor allows you to create and manage process pools in Python. Although the ProcessPoolExecutor has been available since Python 3.2, it is not widely used, perhaps because of misunderstandings of the capabilities and limitations of Processes and Threads in Python.
Yes, it's thread-safe.
Python multiprocessing Pool can be used for parallel execution of a function across multiple input values, distributing the input data across processes (data parallelism).
As stated in the documentation, concurrent.futures.ProcessPoolExecutor is a wrapper around a multiprocessing.Pool. As such, the same limitations of multiprocessing apply (e.g. objects need to be pickleable).
However, concurrent.futures aims to provide an abstract interface that can be used to manage different types of asynchronous tasks in a convenient way. e.g. changing your async strategy from using process pools to using threads is frequently as simple as changing one or two lines of code (rather than needing to code it all up yourself). Another (related) benefit in the abstraction is that concurrent.futures provides a single API to remember -- And you can pick the Executor that is most suited for the task. Is using your process IO bound? Awesome, use a ThreadPoolExecutor. Are you going to have trouble speeding things up because of the Global Interpreter Lock (GIL)? No problem, use a ProcessPoolExecutor.
90
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
