How do I change a value in a .npz file?

Question

I want to change one value in an npz file.

The npz file contains several npy's, I want all but one ( 'run_param' ) to remain unchanged and I want to save over the original file.

This is my working code:

DATA_DIR = 'C:\\Projects\\Test\\data\\'
ass_file = np.load( DATA_DIR + 'assumption.npz' )
run_param = ass_file['run_param']

print ass_file['run_param'][0]['RUN_MODE']
ass_file['run_param'][0]['RUN_MODE'] = 1        (has no effect)
print ass_file['run_param'][0]['RUN_MODE']

print run_param[0]['RUN_MODE']
run_param[0]['RUN_MODE'] = 1
print run_param[0]['RUN_MODE']

This produces:

0
0
0
1

I can't seem to change the value in the original npy.

My code to save afterward is:

np.savez( DATA_DIR + 'assumption.npz', **ass_file )   #
ass_file.close()

How to make this work?

Answer 1

Why your code did not work

What you get from np.load is a NpzFile, which may look like a dictionary but isn't. Every time you access one if its items, it reads the array from file, and returns a new object. To demonstrate:

>>> import io
>>> import numpy as np
>>> tfile = io.BytesIO()  # create an in-memory tempfile
>>> np.savez(tfile, test_data=np.eye(3))  # save an array to it
>>> tfile.seek(0)  # to read the file from the start
0
>>> npzfile = np.load(tfile)
>>> npzfile['test_data']
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
>>> id(npzfile['test_data'])
65236224
>>> id(npzfile['test_data'])
65236384
>>> id(npzfile['test_data'])
65236704

The id function for the same object is always the same. From the Python 3 Manual:

id(object) Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. ...

This means that in our case, each time we call npz['test_data'] we get a new object. This "lazy reading" is done to preserve memory and to read only the required arrays. In your code, you modified this object, but then discarded it and read a new one later.

---

So what can we do?

If the npzfile is this weird NpzFile instead of a dictionary, we can simply convert it to a dictionary:

>>> mutable_file = dict(npzfile)
>>> mutable_file['test_data'][0,0] = 42
>>> mutable_file
{'test_data': array([[ 42.,   0.,   0.],
                     [  0.,   1.,   0.],
                     [  0.,   0.,   1.]])}

You can edit the dictionary at will and save it.

Answer 2

Using numpy.savez with **kwds, the arrays are saved with the keyword names.

   >>> outfile = TemporaryFile()
   >>> np.savez(outfile, x=x, y=y)
   >>> outfile.seek(0)
   >>> npzfile = np.load(outfile)
   >>> npzfile.files
   ['y', 'x']
   >>> npzfile['x']
   array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

while using savez with "just" *args, the arrays are saved with default names.

 >>> np.savez(outfile, x, y)
 >>> outfile.seek(0) # Only needed here to simulate closing & reopening file
 >>> npzfile = np.load(outfile)
 >>> npzfile.files
 ['arr_1', 'arr_0']
 >>> npzfile['arr_0']
 array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

Re-read at least the docstring numpy help and use the proposed syntax.

   print numpy.savez.__doc__

Answer 3

In case you have multiple fields

You might want to preserve the original structure this is a simple workflow for doing this.

Read the file to dict

filename = "file.npz"
filedic = dict(np.load(filename))

Modify the entry

filedic['myentry'] = mynewvalue

Write the file back

np.savez(filename, **filedic)