Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

numpy fromfile(count = -1) returns array of zeros on Mac OS for huge filesize

I am reading a file using numpy.fromfile:

mat1=numpy.fromfile("path/to/file", numpy.uint8, 40000, "")

this reads the file as I expect.

but when I read the entire file:

mat1=numpy.fromfile("path/to/file", numpy.uint8, -1, "")

this gives me an array of zeros. [0,0,0,...,0,0,0]

I tired: numpy.count_nonzeros(mat1) which gives 0

size(mat1) gives the exact size of the file in bytes. So it makes an array of the expected size, but its full of zeros.

like image 466
change Avatar asked Apr 17 '13 22:04

change


People also ask

What is Fromfile in numpy?

fromfile() function. The fromfile() function is used to construct an array from data in a text or binary file. Syntax: numpy.fromfile(file, dtype=float, count=-1, sep='')

How does numpy Fromfile work?

fromfile. Construct an array from data in a text or binary file. A highly efficient way of reading binary data with a known data-type, as well as parsing simply formatted text files.

Are numpy arrays slow?

The reason why NumPy is fast when used right is that its arrays are extremely efficient. They are like C arrays instead of Python lists.


1 Answers

This was apparently mitigated by OS X upgrades. The original issue pointed out by @MichaelJCox was closed, but a patch was never applied:

Closing. The fix is to upgrade to Mavericks.

like image 106
rll Avatar answered Sep 25 '22 22:09

rll