Prevent MPI from busy looping

Question

I have an MPI program which oversubscribes/overcommits its processors. That is: there are many more processes than processors.

Only a few of these processes are active at a given time, though, so there shouldn't be contention for computational resources.

But, much like the flock of seagulls from Finding Nemo, when those processes are waiting for communication they're all busy-looping, asking "Mine? Mine? Mine?"

                                      

I am using both Intel MPI and OpenMPI (for different machines). How can I convince them both not to busy loop?

My quick and dirty solution has been to use MPI_Iprobe in a loop with a sleep command (see here).

Answer 1

Its a been a while since this has been asked but this post may have the answer you're looking for. (tl;dr pass --mca mpi_yield_when_idle 1 as a parameter to mpirun if you're using OpenMPI)

Other than that, if your MPI processes are waiting at MPI Barriers, you can set the I_MPI_WAIT_MODE=1 to prevent the busy loop in INTEL's MPI. for OpenMPI, see linked post.