What does ksize and k mean in cornerHarris?

Question

I was playing around with cornerHarris function in OpenCV. I could not understand what does ksize and k mean in the function. The documentation mentions ksize to be Aperture parameter of Sobel derivative used and k to be Harris detector free parameter in the equation but I am not sure what does it really mean?

Could someone help me understand?

I tried to detect corners in a cube and it came it as :

with the simple code I used from the documentation:

    import cv2
    import numpy as np

    filename = "cube.jpg"

    img = cv2.imread("./images/{}".format(filename))

    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    gray = np.float32(gray)
    dst = cv2.cornerHarris(gray,12,3,0.04)

    dst = cv2.dilate(dst,None)

    # Threshold for an optimal value, it may vary depending on the image.
    img[dst>0.01*dst.max()]=[0,0,255]

    cv2.imshow('dst',img)
    if cv2.waitKey(0) & 0xff == 27:
        cv2.destroyAllWindows()

I tried tweaking K but could not understand the role of it although I realized increasing it beyond a limit resulted in zero corner being detected.

Answer 1

Harris Corner detector is used to extract corners from grayscale images. The Harris detector works by first calculating the image gradient, then calculating the covariance of the gradient, which is an approximation of the local Hessian.

It has 4 main steps:

1. Edge detection (spatial derivative calculation) - The first step is to convert the grayscale image into an image of edges. There are many techniques to do this, but the cv2 uses a filter called Sobel's kernel, which gets cross-correlated with the original image. The ksize parameter determines the size of the Sobel kernel (3x3, 5x5, etc..). As the size increases, more pixels are part of each convolution process and the edges will get more blurry.

2. Structure tensor setup - Basically we construct a matrix M which represents the direction of the gradients (edges) at every point of the image. This matrix can then be used to determine which of the edge pixels are corners:

3. Harris response calculation - In this step, we calculate the "corner score" R of each edge pixel. The idea is that a pixel is defined as a corner only if it has big gradients in 2 perpendicular directions, which means the M matrix has 2 big eigenvalues (1 big eigenvalue will simply be an edge). Here we can see Harris detector's free parameter - k. It is an empirically determined constant in the range [0.04,0.06]:

The k parameter lets you influence in this step, trading off precision and recall. So with a bigger k, you will get less false corners but you will also miss more real corners (high precision), with a smaller k you will get a lot more corners, so you will miss less true corners, but get a lot of false ones (high recall).

4. Non-maximum suppression - The maxima of corner pixels in every local area is found and the rest are suppressed.