error in mpirun command

Question

       --------------------------------------------------------------------------
        MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD 
        with errorcode 1.

        NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
         You may or may not see output from other processes, depending on
         exactly when Open MPI kills them.
           --------------------------------------------------------------------------
          --------------------------------------------------------------------------
        mpirun has exited due to process rank 2 with PID 19175 on
         node mosura15 exiting without calling "finalize". This may
            have caused other processes in the application to be
            terminated by signals sent by mpirun (as reported here).

I am running a simulation. In MPI command, I found the above error. What is reason behind this. How can I resolve this ?

Answer 1

It looks like the 3rd instance of your program (id 2) crashed and didn't call MPI_Finalize() to close down, and so mpirun closed all the other copies of the program as well. Is there something causing that particular node to crash, or is it a different node each time?