Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to overwrite array inside h5 file using h5py

I'm trying to overwrite a numpy array that's a small part of a pretty complicated h5 file.

I'm extracting an array, changing some values, then want to re-insert the array into the h5 file.

I have no problem extracting the array that's nested.

f1 = h5py.File(file_name,'r') X1 = f1['meas/frame1/data'].value f1.close() 

My attempted code looks something like this with no success:

f1 = h5py.File(file_name,'r+') dset = f1.create_dataset('meas/frame1/data', data=X1) f1.close() 

As a sanity check, I executed this in Matlab using the following code, and it worked with no problems.

h5write(file1, '/meas/frame1/data', X1); 

Does anyone have any suggestions on how to do this successfully?

like image 360
user3508433 Avatar asked Apr 07 '14 20:04

user3508433


People also ask

What is h5py File in Python?

An HDF5 file is a container for two kinds of objects: datasets , which are array-like collections of data, and groups , which are folder-like containers that hold datasets and other groups. The most fundamental thing to remember when using h5py is: Groups work like dictionaries, and datasets work like NumPy arrays.

How do I explore HDF5 files?

Open a HDF5/H5 file in HDFView hdf5 file on your computer. Open this file in HDFView. If you click on the name of the HDF5 file in the left hand window of HDFView, you can view metadata for the file.

Why are HDF5 files so large?

This is probably due to your chunk layout - the more chunk sizes are small the more your HDF5 file will be bloated. Try to find an optimal balance between chunk sizes (to solve your use-case properly) and the overhead (size-wise) that they introduce in the HDF5 file.

Can HDF5 store strings?

All strings in HDF5 hold encoded text.You can't store arbitrary binary data in HDF5 strings. Not only will this break, it will break in odd, hard-to-discover ways that will leave you confused and cursing.


2 Answers

You want to assign values, not create a dataset:

f1 = h5py.File(file_name, 'r+')     # open the file data = f1['meas/frame1/data']       # load the data data[...] = X1                      # assign new values to data f1.close()                          # close the file 

To confirm the changes were properly made and saved:

f1 = h5py.File(file_name, 'r') np.allclose(f1['meas/frame1/data'].value, X1) #True 
like image 196
askewchan Avatar answered Sep 17 '22 15:09

askewchan


askewchan's answer describes the way to do it (you cannot create a dataset under a name that already exists, but you can of course modify the dataset's data). Note, however, that the dataset must have the same shape as the data (X1) you are writing to it. If you want to replace the dataset with some other dataset of different shape, you first have to delete it:

del f1['meas/frame1/data'] dset = f1.create_dataset('meas/frame1/data', data=X1) 
like image 34
weatherfrog Avatar answered Sep 17 '22 15:09

weatherfrog