Problem with imgIdx in DMatch class using FlannBasedMatcher in Python

I have the same issue as here:

how to access best image corresponding to best keypoint match using opencv flannbasedmatcher and dmatch

Unfortunately, this post doesn't have an answer.

I have several images (and corresponding descriptors), that I add to the FlannBasedMatcher, using the 'add' method (once for each set of descriptors, corresponding to a single image).

However, when I match an image, the return imgIdx is way larger than the number of images in the training set. I feel like each descriptor is treated as an image, but this is not what I want.

I want to know which image (or set of descriptors) each feature has been matched to.

Here is a part of my code (I simplified it a bit, and I know 'test' is not great for a variable name, but it's temporary). Also here I read .key files, which are basically files containing keypoints and descriptors of an image (extracted with SIFT).

I just precise that in the following code, featMatch is just a class I created to create a FlannBasedMatcher (with initialization parameters).

with open(os.path.join(ROOT_DIR,"images\\descriptor_list.txt"),'r') as f:
    for line in f:
        folder_path = os.path.join(ROOT_DIR,"images\\",line[:-1]+"\\","*.key")
        list_key = glob.glob(folder_path)

        test2 = []
        for key in list_key:
            if os.path.isfile(key):
                feat = Features()
                feat.readFromFile(key)
                test = feat.descriptors
                test2 = test2+test
         featMatch.add(test2)

# Read submitted picture features
feat = Features()
feat.readFromFile(os.path.join(ROOT_DIR,"submitted_picture\\sub.key"))
matches = []
matches.append(featMatch.knnMatch(np.array(feat.descriptors), k=3))
print(matches)

I was expecting, when looking at the matches, and more specifically at the imgIdx of the matches, to be told which image index the matching feature (trainIdx) correspond to, based on the number of descriptor sets I added with 'add' method.

But following this assumption, I should be able to have imgIdx larger than the number of images (or training sets) in my training set.

However, here, I get numbers such as 2960, while I only have about 5 images in my training set.

My guess is that it returns the feature index instead of the image index, but I don't know why.

I noticed that the 'add' method in C++ takes an array of array, where we have a list of descriptor sets (one for each image I guess). But here I have a different number of features for each image, so I can't really create a numpy array with a different number of rows in each column.

Thanks.

I finally figure it out after looking at the C++ source code of matcher.cpp:

https://github.com/opencv/opencv/blob/master/modules/features2d/src/matchers.cpp

I'm gonna post the answer, in case somebody needs it someday.

I thought that the 'add' method would increment the image count when called, but it does not. So, I realized that I have to create a list of Mat (or numpy array in python) and give it once to 'add', instead of calling it for each image.

So here is the updated (and working) source code:

with open(os.path.join(ROOT_DIR,"images\\descriptor_list.txt"),'r') as f:
    list_image_descriptors = []
    for line in f:
        folder_path = os.path.join(ROOT_DIR,"images\\",line[:-1]+"\\","*.key")
        list_key = glob.glob(folder_path)

        for key in list_key:
            if os.path.isfile(key):
                feat = Features()
                feat.readFromFile(key)
                img_descriptors = np.array(feat.descriptors)
        list_image_descriptors.append(img_descriptors)
     featMatch.add(list_image_descriptors)

# Read submitted picture features
feat = Features()
feat.readFromFile(os.path.join(ROOT_DIR,"submitted_picture\\sub.key"))
matches = []
matches.append(featMatch.knnMatch(np.array(feat.descriptors), k=3))
print(matches)

Hope this helps.