Do I need to have a corresponding MPI::Irecv for an MPI::Isend?

Question

A seemingly silly question but I can't seem to find a definitive answer one way or the other.

The basic questions is do I need to have a corresponding MPI::Irecv for an MPI::Isend?

That is, even though the message sending is non-blocking, as long as I wait on the sends to complete before reusing the send buffers, do I need to use non-blocking receives & waits to receive the sent buffers?

My point is, I want to use non-blocking sends to “do other stuff” while the message is being sent but the receiver process will use the buffers immediately so I want them to block until the buffer is truly received.

It seems like I should be able to receive messages with MPI::Recv even though they were sent with MPI::Isend but I am wondering if I am missing something?

A bit of simple pseudo code

  if( rank == 0 ){
   int r;
   for ( int i = 0; i < n; i++ ){

     // DO SOME STUFF HERE...

     request.Wait(status);
     request2.Wait(status);
     request3.Wait(status);

     r = i;
     memcpy( key, fromKey(i), ...);
     memcpy( trace, fromTrace(i), ...);

     request  = MPI::COMM_WORLD.Isend( &r, 1, MPI::INT, node, tag );
     request2 = MPI::COMM_WORLD.Isend( key, 10, MPI::INT, node, tag );
     request3 = MPI::COMM_WORLD.Isend( trace, nBytesTotal, MPI::BYTE, node, tag );

     // DO SOME MORE STUFF HERE.

   }
   r = -1;
   request  = MPI::COMM_WORLD.Isend( &r, 1, MPI::INT, node, tag );

   // Carry on ...

  } else {

   int r = -1;
   MPI::COMM_WORLD.Recv( &r, 1, MPI::INT, 0, tag, status );
   while( r >= 0 ){

     MPI::COMM_WORLD.Recv( &key, 10, MPI::INT, 0, tag, status );
     memcpy( saveKey, key, ...);

     MPI::COMM_WORLD.Recv( &trace, nBytesTotal, MPI::BYTE, 0, tag, status );
     memcpy( saveTrace, trace, ...);

     MPI::COMM_WORLD.Recv( &r, 1, MPI::INT, 0, tag, status );
  }

Answer 1

No, you can freely mix blocking and non-blocking MPI operations on both ends of the communication. Blocking is related to when the MPI call returns control to your code and not to the content of the message(s) being transmitted.

Every MPI message carries an "envelope" with itself, that contains its source, destination, tag, and communicator. To successfully receive a message your receive operation should only match its envelope. The envelope in no way specifies how exactly was the message sent - was it via a blocking, was it via a non-blocking operation, was it a synchronous send (MPI_Ssend) or a buffered one (MPI_Bsend). The only exception is the so-called "ready mode" send that is initiated with MPI_Rsend() or MPI_Irsend() which requires that the matching receive operation has already been posted or the message will not be delivered.

That's why the term "matching receive operation" is used throughout the MPI standard and not something like "corresponding receive function".

Answer 2

Yes, this is fine. There is no requirement that the send/recv "kinds" (or whatever you want to call them) match.

Answer 3

This is no matter what send/recv you use; but the implementation is important. You must be aware of blocking points in your code. For example In using blocking communication you must be care about send and receive calls for example look at this code

 if(rank==0)
 {
 MPI_Send(x to process 1)
 MPI_Recv(y from process 1)
 }
 if(rank==1)
 {
 MPI_Send(y to process 0);
 MPI_Recv(x from process 0);
 }

what happens in this case? process 0 sends x to process 1 and blocks until process 1 receives x, process 0 sends y to process 0 and blocks until process 0 receives y, but process 0 blocked so process 1 blocks for infinity until you kill the 2 processes