pandas HDFStore - how to reopen?

Question

I created a file by using:

store = pd.HDFStore('/home/.../data.h5')

and stored some tables using:

store['firstSet'] = df1
store.close()

I closed down python and reopened in a fresh environment.

How do I reopen this file?

When I go:

store = pd.HDFStore('/home/.../data.h5')

I get the following error.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 207, in __init__
    self.open(mode=mode, warn=False)
  File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 302, in open
    self.handle = _tables().openFile(self.path, self.mode)
  File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 230, in openFile
    return File(filename, mode, title, rootUEP, filters, **kwargs)
  File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 495, in __init__
    self._g_new(filename, mode, **params)
  File "hdf5Extension.pyx", line 317, in tables.hdf5Extension.File._g_new (tables/hdf5Extension.c:3039)
tables.exceptions.HDF5ExtError: HDF5 error back trace

  File "H5F.c", line 1582, in H5Fopen
    unable to open file
  File "H5F.c", line 1373, in H5F_open
    unable to read superblock
  File "H5Fsuper.c", line 334, in H5F_super_read
    unable to find file signature
  File "H5Fsuper.c", line 155, in H5F_locate_signature
    unable to find a valid file signature

End of HDF5 error back trace

Unable to open/create file '/home/.../data.h5'

What am I doing wrong here? Thank you.

Answer 1

In my hands, following approach works best:

df = pd.DataFrame(...)

"write"
with pd.HDFStore('test.h5',  mode='w') as store:
    store.append('df', df, data_columns= df.columns, format='table')

"read"
with pd.HDFStore('test.h5',  mode='r') as newstore:
    df_restored = newstore.select('df')

Answer 2

You could try doing instead:

store = pd.io.pytables.HDFStore('/home/.../data.h5')
df1 = store['firstSet']

or use the read method directly:

df1 = pd.read_hdf('/home/.../data.h5', 'firstSet')

Either way, you should have pandas 0.12.0 or higher...

Answer 3

I had the same problem and finally fixed it by installing the pytables module (next to the pandas modules which I was using):

conda install pytables

which got me numexpr-2.4.3 and pytables-3.2.0

After that it worked. I am using pandas 0.16.2 under python 2.7.9