Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I read a python pickle database/file from C?

Tags:

I am working on integrating with several music players. At the moment my favorite is exaile.

In the new version they are migrating the database format from SQLite3 to an internal Pickle format. I wanted to know if there is a way to access pickle format files without having to reverse engineer the format by hand.

I know there is the cPickle python module, but I am unaware if it is callable directly from C.

like image 613
Phillip Whelan Avatar asked Aug 18 '09 20:08

Phillip Whelan


People also ask

How do I view a python pickle file?

We can use the pandas library to read a pickle file in Python. The pandas module has a read_pickle() method that can be used to read a pickle file. This method accepts a filepath_or_buffer argument: the file path, the URL, or the buffer from where the pickle file will be loaded.

How do I open a pickle dataset?

Pickling Files To use pickle, start by importing it in Python. To pickle this dictionary, you first need to specify the name of the file you will write it to, which is dogs in this case. Note that the file does not have an extension. To open the file for writing, simply use the open() function.

Which method is used to read a pickle a file?

load() – This function is used to read a pickled object representation from the open file object file and return the reconstituted object hierarchy specified.

How do I extract a pickle file?

To retrieve pickled data, the steps are quite simple. You have to use pickle. load() function to do that. The primary argument of pickle load function is the file object that you get by opening the file in read-binary (rb) mode.


2 Answers

http://www.picklingtools.com/

There is a library called the PicklingTools which I help maintain which might be useful: it allows you to form data structures in C++ that you can then pickle/unpickle ... it is C++, not C, but that shouldn't be a problem these days (assuming you are using the gcc/g++ suite).

The library is a plain C++ library (there are examples of C++ and Python within the distribution showing how to use the library over sockets and files from both C++ and Python), but in general, the basics of pickling to files is available.

The basic idea is that the PicklingTools library gives you "python-like" data structures from C++ so that you can then serialize and deserialize to/from Python/C++. All (?) the basic types: int, long int,string, None, complex, dictionarys, lists, ordered dictionaries and tuples are supported. There are few hooks to do custom classes, but that part is a bit immature: the rest of the library is pretty stable and has been active for 8 (?) years.

Simple example:

#include "chooseser.h" int main() {   Val a_dict = Tab("{ 'a':1, 'b':[1,2.2,'three'], 'c':None }");   cout << a_dict["b"][0];  // value of 1    // Dump to a file   DumpValToFile(a_dict, "example.p0", SERIALIZE_P0);    // .. from Python, can load the dictionary with pickle.load(file('example.p0'))    // Get the result back   Val result;   LoadValFromFile(result, "example.p0", SERIALIZE_P0);   cout << result << endl; } 

There is further documentation (FAQ and User's Guide) on the web site.

Hope this is useful:

Gooday,

Richie

http://www.picklingtools.com/

like image 102
rts1 Avatar answered Sep 18 '22 14:09

rts1


Like Cristian told, you can rather easily embed python code in your C code, see the example here.

Using cPickle is dead easy as well on python you could use something like:

import cPickle  f = open('dbfile', 'rb') db = cPickle.load(f) f.close() # handle db integration f = open('dbfile', 'wb') cPickle.dump(db, f) f.close() 
like image 28
hhurtta Avatar answered Sep 22 '22 14:09

hhurtta