Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Peek the number of rows in an hdf5 file in pandas

I was wondering if there was a way of easily, quickly, and without loading the entire file, getting the number of rows in an hdf5 file, created using pandas, with pandas?

Thank you in advance!

like image 568
Cenoc Avatar asked Oct 20 '14 12:10

Cenoc


People also ask

Can pandas read HDF5?

Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the “fixed” format.

What is HDF in pandas?

Write the contained data to an HDF5 file using HDFStore. Hierarchical Data Format (HDF) is self-describing, allowing an application to interpret the structure and contents of a file with no outside information. One HDF file can hold a mix of related objects which can be accessed as a group or as individual objects.


1 Answers

In [1]: DataFrame(np.random.randn(10,10)).to_hdf('test.h5','df',mode='w',format='table')

In [3]: store = pd.HDFStore('test.h5')

In [4]: store
Out[4]: 
<class 'pandas.io.pytables.HDFStore'>
File path: test.h5
/df            frame_table  (typ->appendable,nrows->10,ncols->10,indexers->[index])

In [5]: store.get_storer('df').nrows
Out[5]: 10
like image 159
Jeff Avatar answered Sep 18 '22 16:09

Jeff