How to serialize/encode a custom object to bytes/bytearray?

Question

I have the following python class:

class Header:
  def __init__(self, id, len):
    self.id = id
    self.len = len

h = Header(1, 10)

How can I serialize/encode an instance of this class, h to bytes or bytearray, which can be written, for example to a socket?

To give a little more perspective, I need to write this object to an unix domain socket where a C++ program is listening to receive the above object (it defines the above struct exactly as above, with same number/type of fields). Encoding by pickle.dump(...) does not work.

The C++ struct is:

typedef struct Header {
  uint32_t id;
  uint32_t len;
}

In fact I am able to interface with this C++ program from Go, as follows.

import (
  "bytes"
  "encoding/binary"
)

type Header struct {
  ID uint2
  Len uint32
}

// output of this function is written to the socket opened by C++ and it works!!
func GetHeaderBuf() *bytes.Buffer, error{
  hdrBuf := new(bytes.Buffer)
  hdr := Header{1, 10}
  if err := binary.Write(hdrBuf, binary.LittleEndian, hdr); err != nil {
    return nil, err
  }
  return hdrBuf, nil
}

What I am looking for is the python equivalent of of the Go code line binary.Write(...) above.

Answer 1

This is called Serialization.

In Python, you can use the standard library pickle module which performs automatically the (de-)serialization, or serialize by hand. In that latter case, you decide the individual attributes to encode and the way to encode them. Then the struct module does the actual byte conversion.

pickle way:

data = pickle.dumps(h)
h2 = pickle.loads(data)

manual way:

Lets say that we need 2 bytes to store an id (less then 65636) and 4 bytes to store a len. We could do

data = struct.pack('>hi', h.ID, h.Len)
h2 = Header(*struct.unpack('>hi', data))

Pickling uses an internal format and should only be used between Python application. On the other hand, struct is specialy suited for heterogeneous applications. Here the > says that the integer values should use the so called network order (big) endianness. This eases the process of exchanging values between different architectures.

If the other part uses C language, struct is with no doubt the way to go.