OpenMP & MPI explanation

Question

A few minutes ago I stumbled upon some text, which reminded me of something that has been wondering my mind for a while, but I had nowhere to ask.

So, in hope this may be the place, where people have hands on experience with both, I was wondering if someone could explain what is the difference between OpenMP and MPI ?

I've read the Wikipedia articles in whole, understood them in segments, but am still pondering; for a Fortran programmer who wishes one day to enter the world of paralellism (just learning the basics of OpenMP now), what is the more future-proof way to go ?

I would be grateful on all your comments

Answer 1

OpenMP is primarily for tightly coupled multiprocessing -- i.e., multiple processors on the same machine. It's mostly for things like spinning up a number of threads to execute a loop in parallel.

MPI is primarily for loosely couple multiprocessing -- i.e., a cluster of computers talking to each other via a network. It can be used on a single machine as kind of a degenerate form of a network, but it does relatively little to take advantage of its being a single machine (e.g., having extremely high bandwidth communication between the "nodes").

Edit (in response to comment): for a cluster of 24 machines, MPI becomes the obvious choice. As noted above (and similar to @Mark's comments) OpenMP is primarily for multiple processors that share memory. When you don't have shared memory, MPI becomes the clear choice.

At the same time, assuming you're going to be using multiprocessor machines (is there anything else anymore?) you might want to use OpenMP to spread the load in each machine across all its processors.

Keep in mind, however, that OpenMP is generally quite a lot quicker/easier to put into use than MPI. Depending on how much speedup you need, scaling up instead of out (i.e. a fewer machines with more processors each) can make the software development enough quicker/cheaper that it can be worthwhile even though it rarely gives the lowest price per core.