image-processingcomputer-visionpattern-matchingscipy.ndimage

Use a custom kernel / image filter to find a specific pattern in a 2d array

Given an image im,

>>> np.random.seed(0)
>>> im = np.random.randint(0, 100, (10,5))
>>> im
array([[44, 47, 64, 67, 67],
       [ 9, 83, 21, 36, 87],
       [70, 88, 88, 12, 58],
       [65, 39, 87, 46, 88],
       [81, 37, 25, 77, 72],
       [ 9, 20, 80, 69, 79],
       [47, 64, 82, 99, 88],
       [49, 29, 19, 19, 14],
       [39, 32, 65,  9, 57],
       [32, 31, 74, 23, 35]])

what is the best way to find a specific segment of this image, for instance

>>> im[6:9, 2:5]
array([[82, 99, 88],
       [19, 19, 14],
       [65,  9, 57]])

If the specific combination does not exist (maybe due to noise), I would like to have a similarity measure, which searches for segments with a similar distribution and tells me for each pixel of im, how good the agreement is. For instance something like

array([[0.03726647, 0.14738364, 0.04331007, 0.02704363, 0.0648282 ],
       [0.02993497, 0.04446428, 0.0772978 , 0.1805197 , 0.08999   ],
       [0.12261269, 0.18046972, 0.01985607, 0.19396181, 0.13062801],
       [0.03418192, 0.07163043, 0.15013723, 0.12156613, 0.06500945],
       [0.00768509, 0.12685481, 0.19178985, 0.13055806, 0.12701177],
       [0.19905991, 0.11637007, 0.08287372, 0.0949395 , 0.12470202],
       [0.06760152, 0.13495046, 0.06344035, 0.1556691 , 0.18991421],
       [0.13250537, 0.00271433, 0.12456922, 0.97      , 0.194389  ],
       [0.17563869, 0.10192488, 0.01114294, 0.09023184, 0.00399753],
       [0.08834218, 0.19591735, 0.07188889, 0.09617871, 0.13773224]])

The example code is python. I think there should be a solution correlating a kernel with im. This will have the issue though, that a segment with the same value but scaled, will give a sharper response.

Solution

  • Template matching would be one of the ways to go about it. Of course deep learning/ML can also be used for more complicated matching.

    Most image processing libraries support some sort of matching function which compares a set of 2 image - reference and the one to match. In OpenCV it returns a score which can used to determine a match. The matching method uses various functions that support scale and/or rotation invariant matching. Beware of licensing constraints in the method you plan to use.

    In case the images may not always be exact, you can use standard deviation (StdDev) to allow for permissible deviation and yet classify them into buckets. Histogram matching may also be used depending on the condition of image to be matched (lighting, color can be important, unless you use specific channels). Use of histogram will avoid matching template in its entirety.

    Ref for Template Matching: