Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to save numpy masked array to file

Tags:

python

numpy

What is the most efficient way of saving a numpy masked array? Unfortunately numpy.save doesn't work:

import numpy as np
a = np.ma.zeros((500, 500))
np.save('test', a)

This gives a:

NotImplementedError: Not implemented yet, sorry...

One way seems to be using pickle, but that unfortunately is not very efficient (huge file sizes), and not platform-independent. Also, netcdf4 seems to work, but it has a large overhead just to save a simple array.

Anyone has had this problem before? I'm tempted just to do numpy.save of array.data and another for the mask.

like image 756
tiago Avatar asked Dec 14 '12 10:12

tiago


People also ask

How do you save a masked array?

Masked array can be saved to file with numpy. savez(), but after loading it, the mask is gone.

How do I save an array to a numpy file?

You can save your NumPy arrays to CSV files using the savetxt() function. This function takes a filename and array as arguments and saves the array into CSV format. You must also specify the delimiter; this is the character used to separate each variable in the file, most commonly a comma.

What is numpy masked array?

A masked array is the combination of a standard numpy. ndarray and a mask. A mask is either nomask , indicating that no value of the associated array is invalid, or an array of booleans that determines for each element of the associated array whether the value is valid or not.


2 Answers

The current accepted answer is somewhat obsolete, and badly inefficient if the array being stored is sparse (it relies on uncompressed pickling of the array).

A better way to save/load a masked array would be to use an npz file:

import numpy as np

# Saving masked array 'arr':
np.savez_compressed('test.npz', data=arr.data, mask=arr.mask)

# Loading array back
with np.load('test.npz') as npz:
    arr = np.ma.MaskedArray(**npz)
like image 110
Dave Avatar answered Oct 11 '22 22:10

Dave


import numpy as np
a = np.ma.zeros((500, 500))
a.dump('test')

then read it with

a = np.load('test')
like image 44
eumiro Avatar answered Oct 11 '22 22:10

eumiro