opencvcomputer-visionconv-neural-networksiftorb

How to more accurately compare the characteristics between two images?

I have developed two methods using SIFT and ORB, but it seems to me that the points do not correspond correctly. Am I using these functions wrongly or do I need something different?

orb = cv2.ORB_create()
keypoints_X, descriptor_X = orb.detectAndCompute(car1_gray, None)
keypoints_y, descriptor_y = orb.detectAndCompute(car2_gray, None)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
matches = bf.match(descriptor_X, descriptor_y)
matches = sorted(matches, key = lambda x: x.distance)
result = cv2.drawMatches(car1_gray, keypoints_X, car2_gray, keypoints_y, matches[:10], car2_gray, flags = 2)

sift = cv2.SIFT_create()

keypoints_X, descriptor_X = sift.detectAndCompute(car1_gray, None)
keypoints_y, descriptor_y = sift.detectAndCompute(car2_gray, None)
bf = cv2.BFMatcher()
matches = bf.knnMatch(descriptor_X, descriptor_y, k=2)

bom = []

for m,n in matches:
    if m.distance < 0.75*n.distance:
        bom.append([m])

result = cv2.drawMatchesKnn(car1_gray, keypoints_X, car2_gray, keypoints_y, bom, None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)

Below the result of SIFT and ORB: SIFT and ORB result

Solution

  • Take a look into SuperGlue, graph neural network based feature matching. Although, they do not provide training code, but two pretrained model for indoor, outdoor is available. Links,

    https://github.com/magicleap/SuperGluePretrainedNetwork

    https://psarlin.com/superglue/

    https://arxiv.org/pdf/1911.11763.pdf

    enter image description here