specification driven packet format parsing?

Question

this could a shot in the dark: we have a wire packet format which is in design and in flux. Is there a flexible way to specify and parse the incoming stream instead of harding coding it? Preferably language agnostic ... but not necessarily so. If there is a good reason not to do that, please enlighten me as well.

Thanks

Oliver

Answer 1

There is a wonderful Python library called Construct. This is by far the easiest way to get started quickly.

If it is more important to be language agnostic, you can limit your wire protocol to ASN.1 BER, since it's usually possible to find an encoder/decoder library for these in whatever language. For example, there's CoDec in Java and PyASN1 in Python. Warning: though powerful, ASN is laden with design-by-committee standardese and it's very hard to get started in it. This site will help.

There are also newer, lighter ASN.1-like approaches like Apache Thrift and Google Protocol Buffers.

Answer 2

Google's Protocol Buffers lets you specify what you want to send, then it generates the parsing and serializing code for you.

Thrift is similar, but provides a IPC framework as well. It integrates the parsing and sockets stuff into one package.

I haven't used these libraries much, but both of them are well documented, language agnostic and are released under a permissive license – I'm sure at least one of them would suit your application.