How Do I Properly Declare a ctype Structure + Union in Python?

Question

I'm messing around with making a binary data parser, and while I could fall back on C, I wanted to see if I could use Python for the task.

I have some inkling of how to get this going, and my current implementation looks something like this:

from ctypes import *

class sHeader(Structure):
    _fields_ = [("CC", c_uint8, 4),
            ("AFC", c_uint8, 2),
            ("TSC", c_uint8, 2),
            ("PID", c_uint16, 13),
            ("TP", c_uint16, 1),
            ("PSI", c_uint16, 1),
            ("TEI", c_uint16, 1),
            ("SyncByte", c_uint8)]

class Header(Union):
    _fields_ = [("sData", sTsHeader),
            ("ulData", c_uint32)]

head = Header()
head.ulData = 0xffffffff
print(head.ulData)
print(head.sData.SyncByte)

print(sHeader.SyncByte)
print(sHeader.TEI)
print(sHeader.PSI)
print(sHeader.TP)
print(sHeader.PID)
print(sHeader.TSC)
print(sHeader.AFC)
print(sHeader.CC)


print(sizeof(sHeader))
print(sizeof(c_uint8))
print(sizeof(c_uint16))
print(sizeof(c_uint32))

Which produces this output:

V:\>C:\Python27\python.exe WidiUnpacker.py
0xffffffffL
0x0
<Field type=c_ubyte, ofs=4, size=1>
<Field type=c_ushort, ofs=2:15, bits=1>
<Field type=c_ushort, ofs=2:14, bits=1>
<Field type=c_ushort, ofs=2:13, bits=1>
<Field type=c_ushort, ofs=2:0, bits=13>
<Field type=c_ubyte, ofs=0:6, bits=2>
<Field type=c_ubyte, ofs=0:4, bits=2>
<Field type=c_ubyte, ofs=0:0, bits=4>
6
1
2
4

So... Looks to me like my bytes aren't bytes so much as words. I don't know enough about Python or ctypes to understand why that is, but it's kind of defeating my purpose at the moment. Any ideas?

Answer 1

Your sHeader has a 4 bit field, then a 2 bit field, then a 2 bit field (total 8 bits = 1 byte) ... but then the next item is a c_uint16 which needs to be aligned on a 2-byte boundary and hence skips over a byte and moves to byte 2 before taking 13 bits.

If you don't want that (and apparently you don't), just make everything a c_uint32 or similar:

from ctypes import *

class sHeader(Structure):
    _fields_ = [("CC", c_uint32, 4),
        ("AFC", c_uint32, 2),
        ("TSC", c_uint32, 2),
        ("PID", c_uint32, 13),
        ("TP", c_uint32, 1),
        ("PSI", c_uint32, 1),
        ("TEI", c_uint32, 1),
        ("SyncByte", c_uint32, 8)] # note: added 8 here

print sHeader.PID
print sHeader.SyncByte

results in:

<Field type=c_uint, ofs=0:8, bits=13>
<Field type=c_uint, ofs=0:24, bits=8>

(I picked uint32 because your bit fields add up to 32 bits. I'm using Python 2.7 here, hence no parentheses on the prints.)

Answer 2

You can control the alignment with a _pack_ class attribute:

class sHeader(Structure):
    _pack_ = 1

results in

4294967295
255
<Field type=c_ubyte, ofs=3, size=1>
<Field type=c_ushort, ofs=1:15, bits=1>
<Field type=c_ushort, ofs=1:14, bits=1>
<Field type=c_ushort, ofs=1:13, bits=1>
<Field type=c_ushort, ofs=1:0, bits=13>
<Field type=c_ubyte, ofs=0:6, bits=2>
<Field type=c_ubyte, ofs=0:4, bits=2>
<Field type=c_ubyte, ofs=0:0, bits=4>
4
1
2
4