Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there any reason to use threading.Lock over multiprocessing.Lock?

If a software project supports a version of Python that multiprocessing has been backported to, is there any reason to use threading.Lock over multiprocessing.Lock? Would a multiprocessing lock not be thread safe as well?

For that matter, is there a reason to use any synchronization primitives from threading that are also in multiprocessing?

like image 501
Jason Baker Avatar asked Dec 30 '09 14:12

Jason Baker


People also ask

Is multiprocessing better than threading?

Multiprocessing is used to create a more reliable system, whereas multithreading is used to create threads that run parallel to each other. multithreading is quick to create and requires few resources, whereas multiprocessing requires a significant amount of time and specific resources to create.

Is threading faster than multiprocessing?

Multiprocessing outshines threading in cases where the program is CPU intensive and doesn't have to do any IO or user interaction. For example, any program that just crunches numbers will see a massive speedup from multiprocessing; in fact, threading will probably slow it down.

Can you use multithreading and multiprocessing together?

Both multithreading and multiprocessing allow Python code to run concurrently. Only multiprocessing will allow your code to be truly parallel. However, if your code is IO-heavy (like HTTP requests), then multithreading will still probably speed up your code.


2 Answers

The threading module's synchronization primitive are lighter and faster than multiprocessing, due to the lack of dealing with shared semaphores, etc. If you are using threads; use threading's locks. Processes should use multiprocessing's locks.

like image 72
jnoller Avatar answered Oct 02 '22 19:10

jnoller


I would expect the multi-threading synchronization primitives to be quite faster as they can use shared memory area easily. But I suppose you will have to perform speed test to be sure of it. Also, you might have side-effects that are quite unwanted (and unspecified in the doc).

For example, a process-wise lock could very well block all threads of the process. And if it doesn't, releasing a lock might not wake up the threads of the process.

In short, if you want your code to work for sure, you should use the thread-synchronization primitives if you are using threads and the process-synchronization primitives if you are using processes. Otherwise, it might work on your platform only, or even just with your specific version of Python.

like image 44
PierreBdR Avatar answered Oct 02 '22 20:10

PierreBdR