Best and safe way to transfer data over a pipe on different platform

Question

Currently I am facing an issue where i am passing a buffer object over a pipe from x64 arch to x86 arch. The object also contains some pointer values, which is 8 bytes in x64 which the same pointer size on x86 is 4 bytes. Now when i am transmitting the object over pipe then size of it is bit more than what x86 platform was expecting for the same object (because pointer size in here is less). What i could understood from similar post in this forum that i might need to use serialization but i do not know how as i have never used serialization before. Will serialization will solve this problem? I am using C++ with GCC compiler. I want the product would work on all arch (ia64, x64 or x86).

Answer 1

A pointer is an address to a memory location within your local running program*. It is useless to send it to another program, more useless to a program running on another machine, even more useless if the architecture of the other machine is different.

Using serialization in your context means sending the content of what is pointed to by the pointer instead of sending the meaningless pointer itself.

To achieve cross-architecture data sending, the easier is to use text for data transfers. Most if not all of widely used cross-architecture protocols use text: HTTP, IMAP, IRC ...

*: I use program instead of process.

Answer 2

boost serialization is designed specifically to :

Here, we use the term "serialization" to mean the reversible deconstruction of an arbitrary set of C++ data structures to a sequence of bytes. Such a system can be used to reconstitute an equivalent structure in another program context. Depending on the context, this might used implement object persistence, remote parameter passing or other facility. In this system we use the term "archive" to refer to a specific rendering of this stream of bytes. This could be a file of binary data, text data, XML, or some other created by the user of this library.

By the way, use POD structures, and make sure to have use data types of a specific type. For that use predefined types (for example, take a look here)

Answer 3

http://code.google.com/apis/protocolbuffers/

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages – Java, C++, or Python.

Worked for me on x86/64, arm