struct serialization in C and transfer over MPI

Question

I have defined a custom struct which I need to send over to another MPI process using the MPI_Bsend (or MPI_Send).

Here is the struct:

struct car{
  int shifts;
  int topSpeed;
}myCar;

The issue is that apart from primitive types MPI doesn't seem to support direct "transmission" of complex data types like the struct shown above. I've heard that I might have to use "serialization".

How should I approach this and successfully send over myCar to process 5?

Answer 1

Jeremiah is right - MPI_Type_create_struct is the way to go here.

It's important to remember that MPI is a library, not built into the language; so it can't "see" what a structure looks like to serialize it by itself. So to send complex data types, you have to explicitly define its layout. In a language that does have native support for serialization, a set of MPI wrappers can concievably make use of that; mpi4py for instance makes use of python's pickle to transparently send complex data types; but in C, you have to roll up your sleeves and do it yourself.

For your structure, it looks like this:

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
#include <stddef.h>

typedef struct car_s {
        int shifts;
        int topSpeed;
} car;

int main(int argc, char **argv) {

    const int tag = 13;
    int size, rank;

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    if (size < 2) {
        fprintf(stderr,"Requires at least two processes.\n");
        exit(-1);
    }

    /* create a type for struct car */
    const int nitems=2;
    int          blocklengths[2] = {1,1};
    MPI_Datatype types[2] = {MPI_INT, MPI_INT};
    MPI_Datatype mpi_car_type;
    MPI_Aint     offsets[2];

    offsets[0] = offsetof(car, shifts);
    offsets[1] = offsetof(car, topSpeed);

    MPI_Type_create_struct(nitems, blocklengths, offsets, types, &mpi_car_type);
    MPI_Type_commit(&mpi_car_type);

    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    if (rank == 0) {
        car send;
        send.shifts = 4;
        send.topSpeed = 100;

        const int dest = 1;
        MPI_Send(&send,   1, mpi_car_type, dest, tag, MPI_COMM_WORLD);

        printf("Rank %d: sent structure car\n", rank);
    }
    if (rank == 1) {
        MPI_Status status;
        const int src=0;

        car recv;

        MPI_Recv(&recv,   1, mpi_car_type, src, tag, MPI_COMM_WORLD, &status);
        printf("Rank %d: Received: shifts = %d topSpeed = %d\n", rank,
                 recv.shifts, recv.topSpeed);
    }

    MPI_Type_free(&mpi_car_type);
    MPI_Finalize();

    return 0;
}

Answer 2

Although Jonathan Dursi's answer is correct, it is overly complicated. MPI provides simpler and less general type constructors more suitable for your problem. MPI_Type_create_struct is ONLY needed when you have different base types (e.g., an int and a float).

For your example, several better solutions exist:

• Assuming that the two integers are aligned in a contiguous memory area (i.e., like an array of integers), you don't need a derived datatype at all. Just send/receive two elements of type MPI_INT with the address of a variable of type car to be used as the send/receive buffer:

  MPI_Send(&send, 2, MPI_INT, dest, tag, MPI_COMM_WORLD);
  MPI_Recv(&recv, 2, MPI_INT, src, tag, MPI_COMM_WORLD, &status);
  

• If you want to use a derived datatype (e.g., for readability or the fun of it), you can use MPI_Type_contiguous which corresponds to arrays:

  MPI_Type_contiguous(2, MPI_INT, &mpi_car_type);
  

• In case the two integers are aligned differently (most likely not the case, but it is machine dependent and MPI implementations exist for a lot of different platforms), you can use MPI_Type_indexed_block: It takes an array of displacements (like MPI_Type_create_struct), but only one oldtype argument and the block-length of every block is 1 by definition:

  MPI_Aint offsets[2];
  offsets[0] = offsetof(car, shifts) ; //most likely going to be 0 
  offsets[1] = offsetof(car, topSpeed);
  MPI_Type_indexed_block(2, offsets, MPI_INT);
  

While the other solution is semantically correct, it is a lot harder to read and may incur a large performance penalty.

Answer 3

Look at MPI_Type_create_struct to build a custom MPI datatype for your object. An example of using it is at http://beige.ucs.indiana.edu/I590/node100.html.

Answer 4

int MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm)

OpenMPI will send count * sizeof(datatype) contiguous bytes starting at buf to allow sending things like int arrays. For example, if you declare a 10 int array int arr[10], you can send with

MPI_Send(arr, 10, MPI_INT, 1, 0, MPI_COMM_WORLD);

and receive similarly. Since buf is a void pointer we can abuse this to send structs by sending sizeof(my_struct) bytes and casting back as struct on the receiving end. Here is an example:

#include "mpi.h"
#include <stdio.h>

typedef struct 
{
    char a;
    int b;
    short c;
} my_struct;


int main (int argc, char *argv[])
{
    int  numtasks, taskid;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &taskid);
    MPI_Comm_size(MPI_COMM_WORLD, &numtasks);


    if (taskid == 0) 
    {
        my_struct m;
        m.a = '!';
        m.b = 1234;
        m.c = 5678;

        MPI_Send(&m, sizeof(my_struct), MPI_CHAR, 1, 0, MPI_COMM_WORLD);
    }
    else 
    {
        my_struct m;
        MPI_Recv(&m, sizeof(my_struct), MPI_CHAR, 0, 0, MPI_COMM_WORLD, 
                 MPI_STATUS_IGNORE);
        printf("%c %d %d\n", m.a, m.b, m.c); 
    }

    MPI_Finalize();
}

Since C arrays store data contiguously, we can even send arrays of structs similarly to how we malloc an array of structs. So if you had a my_struct m_array[10] you would send (and receive similarly) with

MPI_Send(m_array, sizeof(my_struct) * 10, MPI_CHAR, 1, 0, MPI_COMM_WORLD);