How do I create KeyPoints to compute SIFT?

Question

I am using OpenCV-Python.

I have identified corner points using cv2.cornerHarris. The output is of type dst.

I need to compute SIFT features of the corner points. The input to sift.compute() has to be of the type KeyPoint.

I'm not able to figure out how to use cv2.KeyPoint().

How do I do this?

Answer 1

Harris detector return dst, which have same shape with your image. Harris mark on dst where it think the corner. So, you have to extract keypoint from dst.

def harris(self, img):
    '''
    Harris detector
    :param img: an color image
    :return: keypoint, image with feature marked corner
    '''

    gray_img = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    gray_img = np.float32(gray_img)
    dst = cv2.cornerHarris(gray_img, 2, 3, 0.04)
    result_img = img.copy() # deep copy image

    # Threshold for an optimal value, it may vary depending on the image.
    result_img[dst > 0.01 * dst.max()] = [0, 0, 255]

    # for each dst larger than threshold, make a keypoint out of it
    keypoints = np.argwhere(dst > 0.01 * dst.max())
    keypoints = [cv2.KeyPoint(x[1], x[0], 1) for x in keypoints]

    return (keypoints, result_img)

Answer 2

I think you totally got it wrong. The output is of type "dst" --> Please be mindful that dst which is returned by the function cv2.cornerHarris is a floating point Mat containing the Harris corners detected in the image.

A example of code which I used in python is here for computing corners in a image. You can use the return data and convert it to KeyPoints type. Note that key point structure is defined as OpenCV KeyPoint Structure and each key point is specified by Image space 2d coordinates of type Point2f. Just convert each detected corners to Point2f and use it for your sift function.

#sample code to read image and estimate the harris corner. 
import cv2
import numpy as np


def cvComputeHarrisCorner(img):                                                             
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)                                             
    gray = np.float32(gray)                                                                 
    dst = cv2.cornerHarris(gray,2,3,0.04)                                                   
    dst = cv2.dilate(dst,None)                                                              
    img[dst>0.01*dst.max()]=[0,0,255]                                                       
    return img                                                                              


def main():                                                                                 
   img = cv2.imread('img.jpg')                                  
   cvComputeHarrisCorner(img)                                                                     
   cv2.imshow('dst',img)                                                                   
   if cv2.waitKey(0) & 0xff == 27:                                                         
        cv2.destroyAllWindows()                                                             

if __name__ == "__main__":                                                   
   main()                                                                                  

Instead of explaining all what you need here, I would direct you to this OpenCV Python tutorial which is very well written and explained in the trunk. Please go through them and you will gradually learn about the concept.

OpenCV Python Tutorials