Terminating all processes with MPI

Question

I'm using MPI to search for a solution in such a way that I divide the problem space between the different threads. Each thread is going through a for loop, and each iteration is a candidate for a solution.

The problem is, when one thread finds the solution, I want it to notify the other threads and they should all terminate immediately (or at least at the end of their current iteration - or the beginning of the next one).

How can I do this with MPI?

Answer 1

You can use MPI_Abort(MPI_COMM_WORLD) to completely shut down everything then and there. A more controlled solution would be for a process to post a nonblocking send with a designated tag to every other process when it finds a solution, and each process checks at the end of an iteration with a nonblocking receive whether such a message has been posted by anyone.

Answer 2

MPI doesn't have a lot in the way of "push" notifications, so you can't really force the other processes (not threads; an important distinction in this case!) to know that something's happened.

@pmg is right, you can update a flag that everyone can see. How to do that with MPI-2 "one-sided messages" is described with code at another question: Creating a counter that stays synchronized across MPI processes . You could use that approach and just have everyone check the counter before continuing on their batch of processes. Note though that this is a lot of network traffic for every process for every iteration! Another approach would just be for every few iterations to do an allreduce or something similar to see if anyone has hit the solution. That at least is a bit more optimized in terms of bandwidth, but only works well if the iterations are likely to be more or less synchronous. Still another approach would be for a process to send messages to all the others if they have found the answer, and to test for such a message every iteration (or every several).