How to manage MIN_MATCH_COUNT in image comparison using canny?

I am using canny for image comparison. I am getting correct results for matching and non matching objects after comparing using canny image edging. At times it is not giving the right result and for that I need to keep changing the MIN_MATCH_COUNT. Any solution to keep the MIN_MATCH_COUNT and canny should compare each and every edge of the image.

MIN_MATCH_COUNT = 20

img1 = canny.copy()
img2 = canny1.copy()


# Initiate SIFT detector
sift = cv.SIFT_create()
# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)
flann = cv.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1,des2,k=2)
# store all the good matches as per Lowe's ratio test.
good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)
if len(good)>MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)
    M, mask = cv.findHomography(src_pts, dst_pts, cv.RANSAC,5.0)
    matchesMask = mask.ravel().tolist()
    h,w = img1.shape
    pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)
    dst = cv.perspectiveTransform(pts,M)
    img2 = cv.polylines(img2,[np.int32(dst)],True,255,3, cv.LINE_AA)
    print ("Both the images are matching")
else:
    print( "Not enough matches are found and hence images are not same - {}/{} and hence both the images are not matching".format(len(good), MIN_MATCH_COUNT) )
    matchesMask = None
draw_params = dict(matchColor = (0,255,0), # draw matches in green color 
                   singlePointColor = None,
                   matchesMask = matchesMask, # draw only inliers
                   flags = 2)
img3 = cv.drawMatches(img1,kp1,img2,kp2,good,None,**draw_params)
plt.imshow(img3, 'gray'),plt.show()   

Below is the image where I am getting result as not matching when MIN_MATCH_COUNT is 20 and if I change it to 9 then it will say images are matching.

Similarly in below image also actual ridges of keys are not matching but its still giving the images are matching without considering the matching point.

You could use a relative criteria, so instead of using an asbolute value for MIN_MATCH_COUNT you can use the percentage of matching keypoints over the total number of keypoints of your model. In this way you can set a threshold based on your specific test, let's say..30% (IDK, just an example). That's what I do in a similar issue. Something like:

matching = len(good)/len(kp1)*100

In this way 0<matching<100 and represent a percentage of similarity, so you can do:

min_threshold = 40

if matching > min_threshold:
...