How can I struct.unpack many numbers at once

Question

I want to put a bunch of packed integers into a file, e.g.:

for i in int_list:
    fp.write(struct.pack('<I', i))

Now I'd like to read them out into int_list. I could do this, but it seems inefficient:

data = fp.read()
int_list = []
for i in xrange(0, len(data), 4):
    int_list.append(struct.unpack('<I', data[i:i+4])[0])

Is there a more efficient way to do this?

Answer 1

You can do it more efficiently in both directions:

>>> import struct

>>> int_list = [0, 1, 258, 32768]
>>> fmt = "<%dI" % len(int_list)
>>> data = struct.pack(fmt, *int_list)
>>> data
'\x00\x00\x00\x00\x01\x00\x00\x00\x02\x01\x00\x00\x00\x80\x00\x00'

>>> # f.write(data)
... # data = f.read()
...

>>> fmt = "<%dI" % (len(data) // 4)
>>> new_list = list(struct.unpack(fmt, data))
>>> new_list
[0, 1, 258, 32768]

Answer 2

array.array should be fast for this. You can specify the type of elements it contains - there are a few for integers (although IIUC only in machine endianness), and then use its fromfile method to read directly from a file.