The purpose of Remote Memory Access (RMA)

Question

I read MPI standard and found a section with MPI_Put and MPI_Get operations. However, it is not clear for me what benefits gives RMA that lays in the foundation of those functions. When it is better to use such functions instead of asynchronous MPI_Isend, MPI_Irecv? What is a general purpose of use RMA (not in MPI context)?

Answer 1

The largest perk of One-Sided communications is the ability to push or fetch data between nodes without the overhead of send/receive.

Imagine a scenario with 2 nodes. You want to push data from node 1 to node 2.

When node 1 sends data to node 2 using MPI_ISend or MPI_Send, node 2 must still formally receive that data using MPI_Irecv. Node 2 is going to wait on that receive, match the MPI tags, and then allow you to continue. This is similar in theory to sockets in that both sides have to coordinate. Even with async versions MPI_I*, there is still coordination.

With MPI_Put, you push the data from node 1 and node 2 will have it. There are synchronization steps involved, but the punchline is that node 2 will have it and not need to formally accept. All nodes in the group have access to the memory buffers attached by the other nodes.

A colleague of mine had a good analogy. MPI_Send and MPI_Recv are like mail delivery where the recipient has to sign for the package. They can wait at the door indefinitely (MPI_Send) or leave a note for you to call back (MPI_ISend).

MPI_Put is like the Amazon delivery person walking into your house and putting the groceries away. MPI_Get is like you walking into a grocery store when it is closed and just grabbing what you need.

Similar to this example, MPI_Put and MPI_Get rely on trust and good design. They should only be used when shared memory and one-direction data movement are effective.

Good Overview: https://www.cc.gatech.edu/~echow/ipcc/hpc-course/19_rma.pdf