Send function pointer via MPI

Question

Is it safe to pass function pointers via MPI as a way of telling another node to call a function? Someone may say that Passing any kind of pointers via MPI is meaningless, but I wrote some code to verify it.

//test.cpp
#include <cstdio>
#include <iostream>
#include <mpi.h>
#include <cstring>

using namespace std;

int f1(int a){return a + 1;}
int f2(int a){return a + 2;}
int f3(int a){return a + 3;}

using F=int (*)(int);

int main(int argc, char *argv[]){
    MPI_Init(&argc, &argv);
    int rank, size;
    MPI_Status state;
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    //test
    char data[10];
    if( 0 == rank ){
        *(reinterpret_cast<F*>(data))=&f2;
        for(int i = 1 ; i < size ; ++i)
            MPI_Send(data, 8, MPI_CHAR, i, 0, MPI_COMM_WORLD);
    }else{
        MPI_Recv(data, 8, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &state);
        F* fp = reinterpret_cast<F*>(data);
        int ans = (**fp)(10);
        cout << ans << endl;
    }


    MPI_Finalize();
    return 0;
}

Here is the output:

12
12
12
12
12
12
12
12
12

I ran it via MVAPICH, and it works well. But I just don't now why since separate address spaces means that the pointer value is USELESS in any process other than the one that generated it.

P.S. here is my hostfile

blade11:1
blade12:1
blade13:1
blade14:1
blade15:1
blade16:1
blade17:1
blade18:2
blade19:1

and I ran mpiexec -n 10 -f hostfile ./test, and compiled it using C++11

Answer 1

You are lucky in the sense that your cluster environment is homogeneous and no address space randomisation for ordinary executables is in place. As a consequence, all images are loaded at the same base address and laid out similarly in memory, hence functions have the same virtual addresses in all MPI ranks (note that this is rarely true for symbols from dynamically linked libraries as those are usually loaded at random addresses).

If you compile the source twice using different compilers or using the same compiler but with different compiler options, then have some ranks run the first executable and the rest run the second one, the program will definitely crash.

Try this:

$ mpicxx -std=c++11 -O0 -o test_O0 test.cpp
$ mpicxx -std=c++11 -O2 -o test_O2 test.cpp
$ mpiexec -f hostfile -n 5 ./test_O0 : -n 5 ./test_O2
12
12
12
12
<crash>

The different levels of optimisation result in function code of different size in test_O0 and test_O2. Consequently, f2 will no longer have the same virtual address in all ranks. The ranks that run the same executable as rank 0 will print 12, while the rest will segfault.