what is a fast way to output h5py dataset to text?

Question

I am using the h5py python package to read files in HDF5 format. (e.g. somefile.h5) I would like to write the contents of a dataset to a text file.

For example, I would like to create a text file with the following contents: 1,20,31,75,142,324,78,12,3,90,8,21,1

I am able to access the dataset in python using this code:

import h5py
f     = h5py.File('/Users/Me/Desktop/thefile.h5', 'r')
group = f['/level1/level2/level3']
dset  = group['dsetname']

My naive approach is too slow, because my dataset has over 20000 entries:

# write all values to file        
for index in range(len(dset)):
        # do not add comma after last value
        if index == len(dset)-1: txtfile.write(repr(dset[index]))
        else:                    txtfile.write(repr(dset[index])+',')
txtfile.close()
    return None

Is there a faster way to write this to a file? Perhaps I could convert the dataset into a NumPy array or even a Python list, and then use some file-writing tool?

(I could experiment with concatenating the values into a larger string before writing to file, but I'm hoping there's something entirely more elegant)

Answer 1

Building a large string has the huge advantage of saving the need for the goofy "last-time switch" thanks to the excellent join method of strings: to replace your whole loop,

txtfile.write(','.join(repr(item) for item in dset))

I'm not sure how much more elegant you demand your code to be...;-)