Pickling cv2.KeyPoint causes PicklingError

Question

I want to search surfs in all images in a given directory and save their keypoints and descriptors for future use. I decided to use pickle as shown below:

#!/usr/bin/env python
import os
import pickle
import cv2

class Frame:
  def __init__(self, filename):
    surf = cv2.SURF(500, 4, 2, True)
    self.filename = filename
    self.keypoints, self.descriptors = surf.detect(cv2.imread(filename, cv2.CV_LOAD_IMAGE_GRAYSCALE), None, False)

if __name__ == '__main__':

  Fdb = open('db.dat', 'wb')
  base_path = "img/"
  frame_base = []

  for filename in os.listdir(base_path):
    frame_base.append(Frame(base_path+filename))
    print filename

  pickle.dump(frame_base,Fdb,-1)

  Fdb.close()

When I try to execute, I get a following error:

File "src/pickle_test.py", line 23, in <module>
    pickle.dump(frame_base,Fdb,-1)
...
pickle.PicklingError: Can't pickle <type 'cv2.KeyPoint'>: it's not the same object as cv2.KeyPoint

Does anybody know, what does it mean and how to fix it? I am using Python 2.6 and Opencv 2.3.1

Thank you a lot

Answer 1

The problem is that you cannot dump cv2.KeyPoint to a pickle file. I had the same issue, and managed to work around it by essentially serializing and deserializing the keypoints myself before dumping them with Pickle.

So represent every keypoint and its descriptor with a tuple:

temp = (point.pt, point.size, point.angle, point.response, point.octave, 
        point.class_id, desc)       

Append all these points to some list that you then dump with Pickle.

Then when you want to retrieve the data again, load all the data with Pickle:

temp_feature = cv2.KeyPoint(x=point[0][0],y=point[0][1],_size=point[1], _angle=point[2], 
                            _response=point[3], _octave=point[4], _class_id=point[5]) 
temp_descriptor = point[6]

Create a cv2.KeyPoint from this data using the above code, and you can then use these points to construct a list of features.

I suspect there is a neater way to do this, but the above works fine (and fast) for me. You might have to play around with your data format a bit, as my features are stored in format-specific lists. I tried to present the above using my idea at its generic base. I hope that this may help you.

Answer 2

Part of the issue is cv2.KeyPoint is a function in python that returns a cv2.KeyPoint object. Pickle is getting confused because, literally, "<type 'cv2.KeyPoint'> [is] not the same object as cv2.KeyPoint". That is, cv2.KeyPoint is a function object, while the type was cv2.KeyPoint. Why OpenCV is like that, I can only make guesses at unless I go digging. I have a feeling it has something to do with it being a wrapper around a C/C++ library.

Python does give you the ability to fix this yourself. I found the inspiration on this post about pickling methods of classes.

I actually use this clip of code, highly modified from the original in the post

import copyreg
import cv2

def _pickle_keypoints(point):
    return cv2.KeyPoint, (*point.pt, point.size, point.angle,
                          point.response, point.octave, point.class_id)

copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoints)

Key points of note:

• In Python 2, you need to use copy_reg instead of copyreg and point.pt[0], point.pt[1] instead of *point.pt.
• You can't directly access the cv2.KeyPoint class for some reason, so you make a temporary object and use that.
• The copyreg patching will use the otherwise problematic cv2.KeyPoint function as I have specified in the output of _pickle_keypoints when unpickling, so we don't need to implement an unpickling routine.
• And to be nauseatingly complete, cv2::KeyPoint::KeyPoint is an overloaded function in C++, but in Python, this isn't exactly a thing. Whereas in the C++, there's a function that takes the point for the first argument, in Python, it would try to interpret that as an int instead. The * unrolls the point into two arguments, x and y to match the only int argument constructor.

I had been using casper's excellent solution until I realized this was possible.

Answer 3

A similar solution to the one provided by Poik. Just call this once before pickling.

def patch_Keypoint_pickiling(self):
    # Create the bundling between class and arguments to save for Keypoint class
    # See : https://stackoverflow.com/questions/50337569/pickle-exception-for-cv2-boost-when-using-multiprocessing/50394788#50394788
    def _pickle_keypoint(keypoint): #  : cv2.KeyPoint
        return cv2.KeyPoint, (
            keypoint.pt[0],
            keypoint.pt[1],
            keypoint.size,
            keypoint.angle,
            keypoint.response,
            keypoint.octave,
            keypoint.class_id,
        )
    # C++ Constructor, notice order of arguments : 
    # KeyPoint (float x, float y, float _size, float _angle=-1, float _response=0, int _octave=0, int _class_id=-1)

    # Apply the bundling to pickle
    copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoint)

More than for the code, this is for the incredibly clear explanation available there : https://stackoverflow.com/a/50394788/11094914

Please note that if you want to expand this idea to other "unpickable" class of openCV, you only need to build a similar function to "_pickle_keypoint". Be sure that you store attributes in the same order as the constructor. You can consider copying the C++ constructor, even in Python, as I did. Mostly C++ and Python constructors seems not to differ too much.

I has issue with the "pt" tuple. However, a C++ constructor exists for X and Y separated coordinates, and thus, allow this fix/workaround.