pythonnumpyimage-processingui-automationgame-automation

OpenCV: An Array of images: How can I identify which image has been found?


I'm learning OpenCV & Python by writing a bot for a 2d spaceship game.

The basic principle is there are certain targets my spaceship can defeat and certain spaceships it cannot defeat. One target is displayed on the screen at a time and you click either next or attack depending on what you want to do. In doing this you're able to cycle through targets until you find one you want to attack.

At the moment my code:

  1. Loads images of ships it cannot defeat into an numpy array
  2. Some pyautigui interactions to skip through menus
  3. Checks the numpy array to determine if it wants to attack or skip the target it's been given
  4. Clicks attack or skip with pyautogui

This all works correctly and the bot happily searches away and either clicks skip or attack.

My issue:

The names of the images being loaded are human-readable e.g. Ship 1.jpg, Ship 2.jpg, Ship 3.jpg When I load them into the numpy array they lose their human-readable component.

My Question:

I want to be able to log what was attacked to refine the detection, at the moment I'm able to output the actual array array([[106, 106, 106, ..., 42, 41, 42], but I can't seem to figure out how to keep the human-readable element. How can I keep the human-readable element?

I'm loading the images like this:

# Load Images 
def loadImages(directory):
    # Intialise empty array
    image_list = []
    # Add images to array
    for i in directory:
        img = cv.imread(i, cv.IMREAD_REDUCED_GRAYSCALE_2)
        image_list.append(img)
    return image_list

And the object detection is done with cv.matchTemplate

def objectDetection(image_list):
    # Object Detection
    for i in image_list:
        needle_img = i
        result = cv.matchTemplate(haystack, needle_img, cv.TM_CCORR_NORMED)

Update with Error from Answer:

I updated my loadImages function with image_list.append((img, i)) and that's worked and I can now see the appended file name attached to the array in the debugger. I also updated the object detection with for i in image_list[i][2]: to select the array object but then I get the following error:

Exception has occurred: UnboundLocalError
local variable 'i' referenced before assignment
  File "C:\Users\test6.py", line 65, in objectDetection
    for i in image_list[i][2]:
  File "C:\Users\test6.py", line 202, in <module>
    objectDetection(ships_to_avoid, 0.9)

Solution

  • You can append tuples to your list. Where the first element is the name and the second is the image.

    image_list.append((img, i))
    

    Now 'i' represents the filename and 'img' the image file. You can now access the name of the image or the image with j being the index (e.g. 0) as follows:

    img_name = image_list[j][1]
    img = image_list[j][0]
    

    Edit 1: if you want to access your images in your list for object detection rewrite it the following way:

    def objectDetection(image_list):
    # Object Detection
        for i in image_list:
            needle_img = i[0]
            needle_name = i[1]
            result = cv.matchTemplate(haystack, needle_img, cv.TM_CCORR_NORMED)
    

    Edit 2: fixed the indices