Using msgpack-python with nested namedtuples

Question

My data structure is like this:

user = UserTuple(
    name=u'Anakin', surname=u'Skywalker', birthdate=datetime.date(1981, 7, 25),
    profile=ProfileTuple(avatar=u'http://localhost/profile.jpg')
)

And I want to pack this data with msgpack-python module. But msgpack converts namedtuples to lists. It can be possible pack data like this with msgpack and preserve namedtuples, just like pickle/cpickle?

Answer 1

You need to have the latest msgpack-python version. v0.4.7 doesn't work. (Currently have to install from master branch).

import msgpack
from collections import namedtuple

User = namedtuple('User', ['name', 'profile'])
Profile = namedtuple('Profile', ['avatar'])

def ext_pack(x):
    if isinstance(x, User):
        return msgpack.ExtType(1, msgpack.packb([x[0], x[1]], default=ext_pack, strict_types=True))
    elif isinstance(x, Profile):
        return msgpack.ExtType(2, msgpack.packb(x[0], default=ext_pack, strict_types=True))
    return x

def ext_unpack(code, data):
    if code == 1:
        name, profile = msgpack.unpackb(data, ext_hook=ext_unpack)
        return User(name, profile)
    elif code == 2:
        avatar = msgpack.unpackb(data, ext_hook=ext_unpack)
        return Profile(avatar)
    return msgpack.ExtType(code, data)

x = User("me", Profile(234))
s = msgpack.packb([x, [1, x]], default=ext_pack, strict_types=True)
msgpack.unpackb(s, ext_hook=ext_unpack)
>>> [User(name='me', profile=Profile(avatar=234)),
 [1, User(name='me', profile=Profile(avatar=234))]]

Here we tag User, Profile as type code 1, 2 respectively. Alternatively, you can treat all namedtuple as the same type code, and store the actual types in the data fields.