How can I read a python pickle database/file from C?

Question

I am working on integrating with several music players. At the moment my favorite is exaile.

In the new version they are migrating the database format from SQLite3 to an internal Pickle format. I wanted to know if there is a way to access pickle format files without having to reverse engineer the format by hand.

I know there is the cPickle python module, but I am unaware if it is callable directly from C.

Answer 1

http://www.picklingtools.com/

There is a library called the PicklingTools which I help maintain which might be useful: it allows you to form data structures in C++ that you can then pickle/unpickle ... it is C++, not C, but that shouldn't be a problem these days (assuming you are using the gcc/g++ suite).

The library is a plain C++ library (there are examples of C++ and Python within the distribution showing how to use the library over sockets and files from both C++ and Python), but in general, the basics of pickling to files is available.

The basic idea is that the PicklingTools library gives you "python-like" data structures from C++ so that you can then serialize and deserialize to/from Python/C++. All (?) the basic types: int, long int,string, None, complex, dictionarys, lists, ordered dictionaries and tuples are supported. There are few hooks to do custom classes, but that part is a bit immature: the rest of the library is pretty stable and has been active for 8 (?) years.

Simple example:

#include "chooseser.h" int main() {   Val a_dict = Tab("{ 'a':1, 'b':[1,2.2,'three'], 'c':None }");   cout << a_dict["b"][0];  // value of 1    // Dump to a file   DumpValToFile(a_dict, "example.p0", SERIALIZE_P0);    // .. from Python, can load the dictionary with pickle.load(file('example.p0'))    // Get the result back   Val result;   LoadValFromFile(result, "example.p0", SERIALIZE_P0);   cout << result << endl; } 

There is further documentation (FAQ and User's Guide) on the web site.

Hope this is useful:

Gooday,

Richie

http://www.picklingtools.com/

Answer 2

Like Cristian told, you can rather easily embed python code in your C code, see the example here.

Using cPickle is dead easy as well on python you could use something like:

import cPickle  f = open('dbfile', 'rb') db = cPickle.load(f) f.close() # handle db integration f = open('dbfile', 'wb') cPickle.dump(db, f) f.close()