Deserializing a heterogeneous map with MessagePack in C++

Question

I’m using MessagePack with C++ and I’m trying to deserialize the equivalent of this Python map:

{'metadata': {'date': '2014-06-25', 'user_id': 501},
 'values': [3.0, 4.0, 5.0],
 'version': 1}

The top-level object is a map with string keys, but the values are of completely different types. My code knows ahead of time what the structure of the object should be; I ought to be able to declare an integer and then tell my deserialization code, “The value of the version key is an integer, so put the value of that integer into this memory address.”

The problem is that I’m not even sure how to get to the point where my C++ code can treat this structure as a map. I would expect to do something like

msgpack::unpacker unpacker;
// ...copy the data into unpacker's buffer...

msgpack::unpacked message;
std::map<std::string, anything> output_map;

unpacker.next(&message);
msgpack::object obj = message.get();
obj.convert(&output_map);

int version_number = output_map.at("version");

Is there any possible type (anything) that would work here? The MessagePack documentation has only trivial examples, and this blog post is better but doesn’t cover this use case.

Answer 1

You can do that using boost::variant. To implement recursive structure, you can use oost::make_recursive_variant as follows:

typedef boost::make_recursive_variant<
    std::string,
    std::map<boost::recursive_variant_, boost::recursive_variant_>,
    std::vector<boost::recursive_variant_>,
    int,
    double
    >::type variant_t;

Here is a documentation: http://www.boost.org/doc/libs/1_55_0/doc/html/variant/tutorial.html#variant.tutorial.recursive.recursive-variant

You also need to write a converter that convert from msgpack::object to variant_t and vice versa as follows:

// Custom converter for variant_t
namespace msgpack {

// Convert from msgpacl::object to variant_t.
inline variant_t& operator>>(object const& o, variant_t& v) {
    switch(o.type) {
    case type::MAP:
        v = std::map<variant_t, variant_t>();
        o.convert(boost::get<std::map<variant_t, variant_t> >(&v));
        break;
    case type::ARRAY:
        v = std::vector<variant_t>();
        o.convert(boost::get<std::vector<variant_t> >(&v));
        break;
    case type::POSITIVE_INTEGER:
        v = int();
        o.convert(boost::get<int>(&v));
        break;
    case type::DOUBLE:
        v = double();
        o.convert(boost::get<double>(&v));
        break;
    case type::RAW:
        v = std::string();
        o.convert(boost::get<std::string>(&v));
        break;
    default:
        break;
    }
    return v;
}


// Convert from variant_t to msgpacl::object.
template <typename Stream>
struct packer_imp:boost::static_visitor<void> {
    template <typename T>
    void operator()(T const& value) const {
        o_.pack(value);
    }
    packer_imp(packer<Stream>& o):o_(o) {}
    packer<Stream>& o_;
};

template <typename Stream>
inline packer<Stream>& operator<< (packer<Stream>& o, const variant_t& v)
{
    boost::apply_visitor(packer_imp<Stream>(o), v);
    return o;
}

} // namespace msgpack

You can get a full example code from gist: https://gist.github.com/redboltz/672c5af16b2907488977 I've used the C++11 feature in the example, so you need to add -std=c++11 option.