How to detect if two images are the same with different cropping ratios?

I have two different images with different dimensions:

100px

and 400px

As you can see the two are clearly the "same" from a human point of view. Now I wanna detect if they are the same. I have been using image magic via the ruby gem called rmagick like so:

img1 = Magick::Image.from_blob(File.read("image_1.jpeg")).first
img2 = Magick::Image.from_blob(File.read("image_2.jpeg")).first

if img1.difference(img2).first < 4000.0 # I have found this to be a good threshold, but does not work for cropped images
  puts "they are the same!!!"
end

While this works well for images that have same ratio/cropping, it is not ideal when they have slightly different cropping and has been resized to the same width.

Is there a way to do it for images with different cropping? I am interested in a solution where I can say something like: One image is contained inside the other and covers somewhere around e.g. 90% of it.

PS. I can get the images in higher resolution if that helps (e.g. the double)

You may want to take a look at feature matching. The idea is to find features in two images and match them. This method is commonly used to find a template (say a logo) in another image. A feature, in essence, can be described as things that humans would find interesting in an image, such as corners or open spaces. There are many types of feature detection techniques out there however my recommendation is to use a scale-invariant feature transform (SIFT) as a feature detection algorithm. SIFT is invariant to image translation, scaling, rotation, partially invariant to illumination changes, and robust to local geometric distortion. This seems to match your specification where the images can have slightly different ratios.

Given your two provided images, here's an attempt to match the features using the FLANN feature matcher. To determine if the two images are the same, we can base it off some predetermined threshold which tracks the number of matches that pass the ratio test described in Distinctive Image Features from Scale-Invariant Keypoints by David G. Lowe. A simple explanation of the test is that the ratio test checks if matches are ambiguous and should be removed, you can treat it as a outlier removal technique. We can count the number of matches that pass this test to determine if the two images are the same. Here's the feature matching results:

Matches: 42

The dots represent all matches detected while the green lines represent the "good matches" that pass the ratio test. If you don't use the ratio test then all the points will be drawn. In this way, you can use this filter as a threshold to only keep the best matched features.

---

I implemented it in Python, I'm not very familiar with Rails. Hope this helps, good luck!

Code

import numpy as np
import cv2

# Load images
image1 = cv2.imread('1.jpg', 0)
image2 = cv2.imread('2.jpg', 0)

# Create the sift object
sift = cv2.xfeatures2d.SIFT_create(700)

# Find keypoints and descriptors directly
kp1, des1 = sift.detectAndCompute(image2, None)
kp2, des2 = sift.detectAndCompute(image1, None)

# FLANN parameters
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks=50)   # or pass empty dictionary
flann = cv2.FlannBasedMatcher(index_params,search_params)
matches = flann.knnMatch(des1,des2,k=2)

# Need to draw only good matches, so create a mask
matchesMask = [[0,0] for i in range(len(matches))]

count = 0
# Ratio test as per Lowe's paper (0.7)
# Modify to change threshold 
for i,(m,n) in enumerate(matches):
    if m.distance < 0.15*n.distance:
        count += 1
        matchesMask[i]=[1,0]

# Draw lines
draw_params = dict(matchColor = (0,255,0),
                   # singlePointColor = (255,0,0),
                   matchesMask = matchesMask,
                   flags = 0)

# Display the matches
result = cv2.drawMatchesKnn(image2,kp1,image1,kp2,matches,None,**draw_params)
print('Matches:', count)
cv2.imshow('result', result)
cv2.waitKey()