Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Appending more datasets into an existing Hdf5 file without deleting other groups and datasets

I have an HDF5 file which contains groups and subgroups inside which there are datasets. I want to open the file and add some datasets to the groups. I took the following approach which is quite simple in python.

    import h5py
    f = h5py.File('filename.h5','w')
    f.create_dataset('/Group1/subgroup1/dataset4', data=pngfile)
    f.close()

The before file looked like this

Before File Image

After File looks like this

After File Image

but I want it to not delete other datasets and groups but rather just append the dataset4 in the line.

like image 727
M. Khubaib-ul-Hassan Khan Avatar asked Jul 04 '16 14:07

M. Khubaib-ul-Hassan Khan


People also ask

Does HDF5 compress data?

One of the most powerful features of HDF5 is its ability to store and modify compressed data. The HDF5 Library comes with two pre-defined compression methods, GNUzip (Gzip) and Szip and has the capability of using third-party compression methods as well.

Why is HDF5 file so large?

This is probably due to your chunk layout - the more chunk sizes are small the more your HDF5 file will be bloated. Try to find an optimal balance between chunk sizes (to solve your use-case properly) and the overhead (size-wise) that they introduce in the HDF5 file.

How are HDF5 files structured?

HDF5 files are organized in a hierarchical structure, with two primary structures: groups and datasets. HDF5 group: a grouping structure containing instances of zero or more groups or datasets, together with supporting metadata. HDF5 dataset: a multidimensional array of data elements, together with supporting metadata.

Can HDF5 store strings?

HDF5 supports two string encodings: ASCII and UTF-8. We recommend using UTF-8 when creating HDF5 files, and this is what h5py does by default with Python str objects.


1 Answers

Just like with the Python open() function, 'w' will truncate any existing file. Use the 'a' mode to add content to the file:

import h5py
f = h5py.File('filename.h5','a')
f.create_dataset('/Group1/subgroup1/dataset4', data=pngfile)
f.close()
like image 101
John Readey Avatar answered Oct 06 '22 23:10

John Readey