I'm trying to match keypoints using open cv. Specifically, I'm using the "sift" detector and "flann" matcher. My code is based on cv2's documentation:
detector = cv2.SIFT_create()
matcher = cv2.FlannBasedMatcher(indexParams=dict(algorithm=0, trees=5), searchParams=dict(checks=50))
kps1, desc1 = detector.detectAndCompute(img1, None)
kps2, desc2 = detector.detectAndCompute(img2, None)
all_matches = matcher.knnMatch(desc1, desc2, 2)
ratio = 0.7
good_matches = []
for m, n in all_matches:
if m.distance <= ratio * n.distance:
good_matches.append(m)
I noticed than even within the good_matches list, I have some keypoints that have more than a single match:
extra_matches = dict()
for match in good_matches:
t_idx = match.trainIdx
reps = [mch for mch in good_matches if mch.trainIdx == t_idx]
if len(reps) > 1 and t_idx not in extra_matches.dict():
extra_matches[t_idx] = reps
print(len(extra_matches)) # not 0
I find this weird because I thought that knnMatch already yields the 2 best matches. Why would I have more than a single match per keypoint after ratio-pruning the matches?
Sure enough, five minutes after posting I found the answer:
FLANN does not do cross-checks, which means I will have repeats of the 2nd keypoints but no repeats for the 1st keypoints (verified in my code as well).
The best practice if you need cross-check with FLANN is to implement your own cross-check or use FLANN to get a subset of descriptors and then use BFMatcher's cross-check option.