Reading binary data into pandas

Question

I have some binary data and I was wondering how I can load that into pandas.

Can I somehow load it specifying the format it is in, and what the individual columns are called?

Edit:
Format is

int, int, int, float, int, int[256] 

each comma separation represents a column in the data, i.e. the last 256 integers is one column.

Answer 1

Even though this is an old question, I was wondering the same thing and I didn't see a solution I liked.

When reading binary data with Python I have found numpy.fromfile or numpy.fromstring to be much faster than using the Python struct module. Binary data with mixed types can be efficiently read into a numpy array, using the methods above, as long as the data format is constant and can be described with a numpy data type object (numpy.dtype).

import numpy as np import pandas as pd  # Create a dtype with the binary data format and the desired column names dt = np.dtype([('a', 'i4'), ('b', 'i4'), ('c', 'i4'), ('d', 'f4'), ('e', 'i4'),                ('f', 'i4', (256,))]) data = np.fromfile(file, dtype=dt) df = pd.DataFrame(data)  # Or if you want to explicitly set the column names df = pd.DataFrame(data, columns=data.dtype.names) 

Edits:

• Removed unnecessary conversion of data.to_list(). Thanks fxx
• Added example of leaving off the columns argument