Use python 2 shelf in python 3

Question

I have data stored in a shelf file created with python 2.7

When I try to access the file from python 3.4, I get an error:

>>> import shelve
>>> population=shelve.open('shelved.shelf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python34\lib\shelve.py", line 239, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "C:\Python34\lib\shelve.py", line 223, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "C:\Python34\lib\dbm\__init__.py", line 88, in open
    raise error[0]("db type could not be determined")
dbm.error: db type could not be determined

I'm still able to access the shelf with no problem in python 2.7, so there seems to be a backward-compatibility issue. Is there any way to directly access the old format with the new python version?

Answer 1

As I understand now, here is the path that lead to my problem:

• The original shelf was created with Python 2 in Windows
• Python 2 Windows defaults to bsddb as the underlying database for shelving, since dbm is not available on the Windows platform
• Python 3 does not ship with bsddb. The underlying database is dumbdbm in Python 3 for Windows.

I at first looked into installing a third party bsddb module for Python 3, but it quickly started to turn into a hassle. It then seemed that it would be a recurring hassle any time I need to use the same shelf file on a new machine. So I decided to convert the file from bsddb to dumbdbm, which both my python 2 and python 3 installations can read.

I ran the following in Python 2, which is the version that contains both bsddb and dumbdbm:

import shelve
import dumbdbm

def dumbdbm_shelve(filename,flag="c"):
    return shelve.Shelf(dumbdbm.open(filename,flag))

out_shelf=dumbdbm_shelve("shelved.dumbdbm.shelf")
in_shelf=shelve.open("shelved.shelf")

key_list=in_shelf.keys()
for key in key_list:
    out_shelf[key]=in_shelf[key]

out_shelf.close()
in_shelf.close()

So far it looks like the dumbdbm.shelf files came out ok, pending a double-check of the contents.