Menu
although I have been playing with pthreads, OpenMP, intel TBB, and threading in general for a while I still don't understand what is the main difference between a message passing interface implementation like OpenMP and a classic threading library it's still unclear to me.
Assuming that writing all the boilerplate code for a threading pool it's not a problem in my case, and I'm using C++, the difference between this 2 technologies boils down to ... ?
I'm also interested in operating with threads over the network while distributing tasks to all the connected machines.
Right now I'm also not considering the limitations in terms of number of platforms supported by OpenMP/OpenMPI because I would like to understand just how this 2 concepts work.
615
MPI and Threads. • MPI describes parallelism between processes. (with separate address spaces) • Thread parallelism provides a shared-memory. model within a process.
The process may be multi-threaded, but only the main thread will make MPI calls (all MPI calls are funneled to the main thread).
MPI, in contrast to OPENMP, does not provide shared memory communication, since MPI is a parallel platform for communication between various machines. MPI enables the transfer of data across threads using the message passing paradigm.
A thread is a lightweight process that can be managed independently by a scheduler. Processes require more time for context switching as they are more heavy. Threads require less time for context switching as they are lighter than processes. Processes are totally independent and don't share memory.
"Classic" threading shares all memory between the threads. This is rather dangerous, since it's very easy to accidentally modify data that another thread might be using, leading to nasty bugs. The onus is on the programmer to carefully protect data against unsafe access. This also (usually) requires all processes to be running on the same machine, with access to the same physical memory.
Using independent processes with a message-passing interface gives you more control over which data is shared and which is private to each process; there is little or no danger of one process unexpectedly modifying another process's state. And as you say, the message passing interface can be generalised to pass messages across a network between processes on separate machines.
191
As a complement to Mike Seymour answer :
The main trade-off depends on what you have to share between your process and threads. With shared memory, you actually share the data between the execution contexts.
With messaging, you need to copy the data to pass it between the execution contexts (threads, processes, processes over several computers).
If your data is small (read: data transmission time is small) compared to the execution time of your context, then MPI should not have a significant overhead compared to shared memory.
At the opposite, if the data to be shared is large (data transmission time) is of the same order of magnitude compared to your execution time, then MPI may not be a good idea.
Last, it you want to cross the boundaries of a single computer, shared memory is out of the game.
23
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
