Reading Structure Binary Data in Python?

Question

Are there any nice Python solutions like Ruby's BinData for reading user-defined binary file/stream formats? If not, then what's the preferred way to this in Python outside of just using the struct module?

I have a binary file that stores "records" of events. The records are dynamic in size, so I must read the first few bytes of each record to determine the record length and record type. Different record types will have different byte layouts. For instance, a record of type "warning" might contain three 4-byte ints, followed by a 128 byte value, while a record of type "info" might just have five 4-byte ints.

It would be nice to define the different record types and their structures in such a way that I could simply pass a binary blob to something, and it handle the rest (object generation, etc). In short, your defining templates/maps on how to interpret binary data.

Answer 1

Maybe you are looking for Construct, a pure-Python 2 & 3 binary parsing library?

Answer 2

Python's struct module works like this:

record_header = struct.Struct("<cb") 
warning = struct.Struct("<iii128")
info = struct.Struct("<iiiii")

while True:
    header_text = input.read(record_header.size)
    # file is empty
    if not header_text:
       break
    packet_type, extra_data = record_header.unpack(header_text)
    if packet_type == 'w':
        warning_data = warning.unpack( input.read(warning.size) )
    elif packet_type == 'i':
        info_data = info.unpack( input.read(info.size) )

See the documentation for details: http://docs.python.org/library/struct.html