C++ JSON Serialization

Question

I want a way to serialize and deserialize Objects to JSON, as automatic as possible.

Serialize: For me, the ideal way is that if I call in an instance JSONSerialize() it returns an string with a JSON object that has all the public properties of the object as "name_of_property": "value". For those values that are primitives, it is straightforward, for objects it should try to call on each JSONSerialize() or ToString() or something like that to recursively serialize all the public properties. For collections it should also behave correctly (just vectors/arrays will be ok).

Deserialize: Just make an instance of the given object (let's say a dog) and call JSONDeserialize(json_string), and that should fill all the public properties, creating the needed objects in case that the properties are not primitives, or the needed collections.

An example should run like that:

Dog *d1 = new Dog(); d1->name = "myDog";  string serialized = d1->JSONSerialize();  Dog *d2 = new Dog(); d2->JSONDeserialize(serialized); std::cout << d2->name; // This will print "myDog" 

Or like that:

Dog *d1 = new Dog(); d1->name = "myDog";  string serialized = JSONSerializer.Serialize(d1);  Dog *d2 = JSONSerializer.Deserialize(serialized, Dog); std::cout << d2->name; // This will print "myDog" 

How can I pull this off easily?

Answer 1

There is no reflection in C++. True. But if the compiler can't provide you the metadata you need, you can provide it yourself.

Let's start by making a property struct:

template<typename Class, typename T> struct PropertyImpl {     constexpr PropertyImpl(T Class::*aMember, const char* aName) : member{aMember}, name{aName} {}      using Type = T;      T Class::*member;     const char* name; };  template<typename Class, typename T> constexpr auto property(T Class::*member, const char* name) {     return PropertyImpl<Class, T>{member, name}; } 

Of course, you also can have a property that takes a setter and getter instead of a pointer to member, and maybe read only properties for calculated value you'd like to serialize. If you use C++17, you can extend it further to make a property that works with lambdas.

Ok, now we have the building block of our compile-time introspection system.

Now in your class Dog, add your metadata:

struct Dog {     std::string barkType;     std::string color;     int weight = 0;          bool operator==(const Dog& rhs) const {         return std::tie(barkType, color, weight) == std::tie(rhs.barkType, rhs.color, rhs.weight);     }          constexpr static auto properties = std::make_tuple(         property(&Dog::barkType, "barkType"),         property(&Dog::color, "color"),         property(&Dog::weight, "weight")     ); }; 

We will need to iterate on that list. To iterate on a tuple, there are many ways, but my preferred one is this:

template <typename T, T... S, typename F> constexpr void for_sequence(std::integer_sequence<T, S...>, F&& f) {     using unpack_t = int[];     (void)unpack_t{(static_cast<void>(f(std::integral_constant<T, S>{})), 0)..., 0}; } 

If C++17 fold expressions are available in your compiler, then for_sequence can be simplified to:

template <typename T, T... S, typename F> constexpr void for_sequence(std::integer_sequence<T, S...>, F&& f) {     (static_cast<void>(f(std::integral_constant<T, S>{})), ...); } 

This will call a function for each constant in the integer sequence.

If this method don't work or gives trouble to your compiler, you can always use the array expansion trick.

Now that you have the desired metadata and tools, you can iterate through the properties to unserialize:

// unserialize function template<typename T> T fromJson(const Json::Value& data) {     T object;      // We first get the number of properties     constexpr auto nbProperties = std::tuple_size<decltype(T::properties)>::value;          // We iterate on the index sequence of size `nbProperties`     for_sequence(std::make_index_sequence<nbProperties>{}, [&](auto i) {         // get the property         constexpr auto property = std::get<i>(T::properties);          // get the type of the property         using Type = typename decltype(property)::Type;          // set the value to the member         // you can also replace `asAny` by `fromJson` to recursively serialize         object.*(property.member) = Json::asAny<Type>(data[property.name]);     });      return object; } 

And for serialize:

template<typename T> Json::Value toJson(const T& object) {     Json::Value data;      // We first get the number of properties     constexpr auto nbProperties = std::tuple_size<decltype(T::properties)>::value;          // We iterate on the index sequence of size `nbProperties`     for_sequence(std::make_index_sequence<nbProperties>{}, [&](auto i) {         // get the property         constexpr auto property = std::get<i>(T::properties);          // set the value to the member         data[property.name] = object.*(property.member);     });      return data; } 

If you want recursive serialization and unserialization, you can replace asAny by fromJson.

Now you can use your functions like this:

Dog dog;  dog.color = "green"; dog.barkType = "whaf"; dog.weight = 30;  Json::Value jsonDog = toJson(dog); // produces {"color":"green", "barkType":"whaf", "weight": 30} auto dog2 = fromJson<Dog>(jsonDog);  std::cout << std::boolalpha << (dog == dog2) << std::endl; // pass the test, both dog are equal! 

Done! No need for run-time reflection, just some C++14 goodness!

This code could benefit from some improvement, and could of course work with C++11 with some adjustments.

Note that one would need to write the asAny function. It's just a function that takes a Json::Value and call the right as... function, or another fromJson.

Here's a complete, working example made from the various code snippet of this answer. Feel free to use it.

As mentioned in the comments, this code won't work with msvc. Please refer to this question if you want a compatible code: Pointer to member: works in GCC but not in VS2015

Answer 2

For that you need reflection in C/C++, which doesn't exist. You need to have some meta data describing the structure of your classes (members, inherited base classes). For the moment C/C++ compilers don't automatically provide that information in built binaries.

I had the same idea in mind, and I used GCC XML project to get this information. It outputs XML data describing class structures. I have built a project and I'm explaining some key points in this page :

Serialization is easy, but we have to deal with complex data structure implementations (std::string, std::map for example) that play with allocated buffers. Deserialization is more complex and you need to rebuild your object with all its members, plus references to vtables ... a painful implementation.

For example you can serialize like this:

    // Random class initialization     com::class1* aObject = new com::class1();      for (int i=0; i<10; i++){             aObject->setData(i,i);     }            aObject->pdata = new char[7];     for (int i=0; i<7; i++){             aObject->pdata[i] = 7-i;     }     // dictionary initialization     cjson::dictionary aDict("./data/dictionary.xml");      // json transformation     std::string aJson = aDict.toJson<com::class1>(aObject);      // print encoded class     cout << aJson << std::endl ; 

To deserialize data it works like this:

    // decode the object     com::class1* aDecodedObject = aDict.fromJson<com::class1>(aJson);      // modify data     aDecodedObject->setData(4,22);      // json transformation     aJson = aDict.toJson<com::class1>(aDecodedObject);         // print encoded class     cout << aJson << std::endl ; 

Ouptuts:

>:~/cjson$ ./main {"_index":54,"_inner":  {"_ident":"test","pi":3.141593},"_name":"first","com::class0::_type":"type","com::class0::data":[0,1,2,3,4,5,6,7,8,9],"com::classb::_ref":"ref","com::classm1::_type":"typem1","com::classm1::pdata":[7,6,5,4,3,2,1]} {"_index":54,"_inner":{"_ident":"test","pi":3.141593},"_name":"first","com::class0::_type":"type","com::class0::data":[0,1,2,3,22,5,6,7,8,9],"com::classb::_ref":"ref","com::classm1::_type":"typem1","com::classm1::pdata":[7,6,5,4,3,2,1]} >:~/cjson$  

Usually these implementations are compiler dependent (ABI Specification for example), and require external descriptions to work (GCCXML output), thus are not really easy to integrate to projects.