How to unpack pkl file?

Question

I have a pkl file from MNIST dataset, which consists of handwritten digit images.

I'd like to take a look at each of those digit images, so I need to unpack the pkl file, except I can't find out how.

Is there a way to unpack/unzip pkl file?

Answer 1

Generally

Your pkl file is, in fact, a serialized pickle file, which means it has been dumped using Python's pickle module.

To un-pickle the data you can:

import pickle   with open('serialized.pkl', 'rb') as f:     data = pickle.load(f) 

For the MNIST data set

Note gzip is only needed if the file is compressed:

import gzip import pickle   with gzip.open('mnist.pkl.gz', 'rb') as f:     train_set, valid_set, test_set = pickle.load(f) 

Where each set can be further divided (i.e. for the training set):

train_x, train_y = train_set 

Those would be the inputs (digits) and outputs (labels) of your sets.

If you want to display the digits:

import matplotlib.cm as cm import matplotlib.pyplot as plt   plt.imshow(train_x[0].reshape((28, 28)), cmap=cm.Greys_r) plt.show() 

The other alternative would be to look at the original data:

http://yann.lecun.com/exdb/mnist/

But that will be harder, as you'll need to create a program to read the binary data in those files. So I recommend you to use Python, and load the data with pickle. As you've seen, it's very easy. ;-)

Answer 2

Handy one-liner

pkl() (   python -c 'import pickle,sys;d=pickle.load(open(sys.argv[1],"rb"));print(d)' "$1" ) pkl my.pkl 

Will print __str__ for the pickled object.

The generic problem of visualizing an object is of course undefined, so if __str__ is not enough, you will need a custom script.