Creating and using a cross platform struct in C++

Question

I am writing a cross platform game with networking capabilities (using SFML and RakNet) and I have come to the point where I have compiled the server on my Ubuntu server and got a client going on my Mac. All the development is done on my Mac so I have initially been testing the server on that, and it has worked fine.

I am sending structs over the network and then simply casting them back from char * to (for example) inet::PlayerAdded. Now this has been working fine (for the most part), but my question is: Will this always work? It seems like a very fragile approach. Will the struct's always be laid out the same even on other platforms, Windows, for example? What would you recommend?

#pragma pack(push, 1)
struct Player
{
    int dir[2];
    int left;
    float depth;
    float elevation;
    float velocity[2];
    char character[50];
    char username[50];
};

// I have been added to the game and my ID is back
struct PlayerAdded: Packet
{
    id_type id;
    Player player;
};
#pragma pack(pop)

Answer 1

This won't work if (among other things) you attempt to do it from little-endian machine to big-endian machine as the correct int representation will be reversed between the two.

This could also fail if the alignment or packing of your structure changes from machine to machine. What if you have some 64-bit machines and some that are 32-bit?

You need to use a proper portable serialization library like Boost.Serialization or Google Protocol Buffers to ensure you have a wire protocol (aka transmissible data format) that can be decoded successfully independent of the hardware.

Once nice thing about Protocol Buffers is that you can compress the data transparently using a ZLIB-compatible stream that is also compatible with the protobuf streams. I have actually done this, it works well. I imagine other decorator streams can be used in an analogous way to enhance or optimize your basic wire protocol as needed.

Answer 2

Like many of the other answers I'd advise against sending raw binary data if it can be avoided. Something like Boost serial or Google Protobuf will do a fine job without too much overhead.

But you certainly can create cross-platform binary structures, it is done all the time and is a very valid way of exchanging data. Layering a "struct" over that data just makes sense. You do however have to be very careful of layout, fortunately most compilers give you many options to do that. "pack" is one such such option and takes care of a lot.

You also need to take care about data sizes. Simple include stdint.h and use the fixed size types like uint32_t. Be careful of floating point values as not all architectures will share the same value, for 32-bit float they likely do. Also for endianess most architectures will use the same, and if they don't you can simply flip it on the client which is different.

Answer 3

The answer to "... laid out the same even on other platforms ..." is generally no. This is so even if such issues as different CPUs and/or different endianness are addressed.

Different operating systems (even on the same hardware platform) might use different data representations; this is normally called the "platform ABI" and it's different between e.g. 32bit/64bit Windows, 32bit/64bit Linux, MacOSX.

'#pragma pack' is only half the way, because beyond alignment restrictions there can be data type size differences. For example, "long" on 64bit Windows is 32bit while it's 64bit on Linux and MacOSX.

That said, the problem obviously isn't new and has been addressed in the past already - the remote procedure call standard (RPC) contains mechanisms for how to define data structures in a platform-independent way and how to encode/decode "buffers" representing these structs. It's called "XDR" (eXternal Data Representation). See RFC1832. As programming goes, this wheel has been reinvented several times; whether you convert to XML, do the low level work yourself with XDR, use google::protobuf, boost or Qt::Variant, there's lot to choose from.

On a purely implementation side: For simpliclty, just assume that "unsigned int" everywhere is 32bit aligned at a 32bit boundary; if you can encode all your data as array of 32bit values then the only externalization issue you have to deal with is endianness.