Reading a binary .dat file as an array

Question

I have a code that goes through several iterations. In each iteration, the code generates a numpy based array. I append the numpy based array to an existing binary .dat file. I use the following code to generate the data:

WholeData = numpy.concatenate((Location,Data),axis=0)  
# Location & Data are two numpy arrays
DataBinary = open('DataBinary.dat','ab')
WholeData.tofile(DataBinary)
DataBinary.close()

I am trying to read the whole binary file into an array. I am having the following difficulties:

1. I tried the following code:

   NewData = numpy.array('f')
   File1 = open('DataBinary.dat','rb')
   NewData.fromstring(File1.read())
   File1.close()
   

   Error status:

   Traceback (most recent call last): File "", line 1, in AttributeError: 'numpy.ndarray' object has no attribute 'fromstring'

2. I tried to use an array based array and then read the file into the array.

   from array import array
   File1 = open('DataBinary.dat','rb')
   NewData.fromstring(File1.read())
   File1.close()
   

However, NewData is erroneous, i.e., it is not same as WholeData. I guess that saving the data as numpy.array and reading it as array.array might not be a good option.

Any suggestion will be appreciated.

Answer 1

I think that numpy.fromfile is what you want here:

import numpy as np
myarray = np.fromfile('BinaryData.dat', dtype=float)

Also note that according to the docs, this is not the best way to store data as "information on precision and endianness is lost". In other words, you need to make sure that the datatype passed to dtype is compatible with what you originally wrote to the file.