opencvcomputer-visionfeature-detectioncorner-detection

What affect does window size have on the results of the harris corner detector in openCV


I am investigating the affect of varying different parameters on the error rate of the Harris corner detector using openCV. The input parameters are window size, size of the kernel of the sobel operator and the value for the k parameter. I have found that when I increase the window size there appears to be an increase in the number of responses per corner. For example, if each window containing a corner is marked by a dot there appears to be a higher density of dots around identified corners when I use a 7x7 window as opposed to a 2x2 window. Changing the window size also appears to increase the number of corners which are correctly identified.

My coding is as follows, which I got from this example

import cv2
import numpy as np

filename = 'resized_image.jpg'
img = cv2.imread(filename)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

gray = np.float32(gray)
dst = cv2.cornerHarris(gray,7,3,0.015)

#result is dilated for marking the corners, not important
dst = cv2.dilate(dst,None)

# Threshold for an optimal value, it may vary depending on the image.
img[dst>0.05*dst.max()]=[0,0,255]
cv2.imwrite('corners3.jpg', img )
cv2.imshow('dst',img)
if cv2.waitKey(0) & 0xff == 27:
    cv2.destroyAllWindows()

Could someone please explain the affect of increasing the window size on the results of the harris corner detector. Specifically why do the number of responses per corner appears to increase with increasing window size. It is my understanding that this function calculates window gradients and performs some smoothing using the sobel operator if this has an effect on results at all. Sorry if this is an obvious question but I am only new to computer vision.


Solution

  • What do you mean by "responses per corner"? It is normal to see multiple clusters of peaks in the detector response in the general vicinity of the "true" corner's location. That happens because the response function is anything but smooth for natural images - after all it computes products of derivatives, which magnifies any "random" oscillations in the original images. All Harris's filter does is to enhance areas where this un-smooth function is "peaky", but the peaks can be close to each other even though your intuition about the image tells you they shouldn't.

    One obvious technique to counter this problem is to compute "smoothed derivatives" (i.e. Gx * I, rather than Ix, where G is a small Gaussian kernel, etc.), but this has the side effect that some "weak" corners may get smoothed out.

    Regardless of whether one uses smoothed derivatives or not, a simple algorithm used to counter this problem and obtain an expected number N of "clean" corners is to select from the response the k * N strongest peaks sorted by decreasing detector response intensity (with k, say, in the 5 to 10 range), and then:

    1. Pick the next peak down the list.
    2. Lookup the Harris detector response value H(x,y) at its location (x, y).
    3. If H(x,y) > 0, then

      Add it to the output clean corner list;

      Set to 0 all values of H in a small m x m neighborhood of (x, y).

    4. Else skip this peak.
    5. Goto 1 if if the size of the output list is less than N and there are any peaks left.

    The width m of the "cluster" suppression neighborhood should express your prior information about how far apart the "real" corners should be from each other. This method obviously assumes that the "best" corner of a cluster is the one with the strongest Harris filter response.

    BTW, if you are into names (or googling them), this is usually called "non-local-maxima suppression".

    BTW-2: See this answer for details on finding all local maxima of the H response, to be then sorted by response value etc.