implementation or ideas for detecting objects in a game

I am trying to figure out a good method to detect the red border in this game with python and struggling getting alot of false positives or missing it completely with my current method. I am currently using open cv and just performing some template matching with small screenshots of the red border. Anyone have any ideas of how to better find this border in this image and see the actual boundries?

Full resolution PNG here: https://ibb.co/FB0MkBh

JPEG here:

Code snippet:

# Define redborder image paths
redborder_image_paths = [
    'bottomLeftBorder.png',
    'bottomRightBorder.png',
    #'topLeftBorder.png',
    'topRightBorder.png',
    'topRightMaxBorder.png',
    'topLeftBorderMaxZoom.png',
    'topleftMaxZoom.png',
    'bottomLeftMaxZoom.png',
    'topRightBorderMaxZoom.png',
    'topRightBorderGreenMap.png',
    'bottomRightBorderGreenMap.png',
    'topRightBorderOutBoundMaxZoom.png',
    'bottomLeftBorderMaxZoomOutBound.png',
    'topleftbordermaxZoomhole.png',
]  

# Calculate the distances from the townhall image to each found redborder image
for border_index, border_gray in enumerate(redborder_images_gray):
    result_border = cv2.matchTemplate(screenshot_gray, border_gray, cv2.TM_CCOEFF_NORMED)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result_border)
    print(f"Border Image caculating.. ({redborder_image_paths[border_index]}): Confidence: {max_val}")

Tried using open CV and not getting great results its definitely sometimes working but often times it finds false positives with high confidence above 8.0 or misses some close borders that it should be able to detect.

I am using openCV to detect certain structures in the image with great results it's mostly this red border line that i'm struggling with. I would only assume there some better logic to detect that I am just too noob to know.

First,I don't think there is an accurate way of doing this. There are too many obstacles and i assume you want to detect the borders for different enemy bases in the game. Therefore for each different base there will be different number of randomly placed obstacles. Theese variatons are to much to generate a stable solution.

However if you want an approximate solution, i did the following:

• Apply very narrow color filtering to detect the yellowish grass inside the red border, because it is easier to detect then red thin border. I assume the color palette do not change much since its a digitally generated image from a game.

• Close the small gaps in the mask with a custom diamond shaped smal sized kernel. Since the borders are not vertical&horizontal standart kernels won't work properly.

For example a 5x5 Kernel looks like this:

• Find small blobs by finding contours on the mask and erase them (fill them black).

• After deleting small blobs close the image again but with a much greater kernel to combine the possible distinct parts of the outer border.

This is the result after color filtering and closing with 7x7 Kernel:

This is the result after finding and erasing the small blob contours:

This is the result after closing with 77x77 Kernel:

Now the borders of the village are finally found and combined we can find its contour as:

In the final result there are various defects around the border, decorations around the borders disturbs the nearby area. Also the small gaps between the lines of borders are not found due to high closing. This method will give you a much better result for a less obstacled village but still it may fail for some of the viilages.

Finally your max sized image's resolution was 3840x2160 px so the kernel sizes are according to this resolution. If you use a lesser image don't forget to reduce the kernel sizes for small and big closing in the processing step.

This is the complete code:

import cv2
import numpy

# Read image
image = cv2.imread('clash.png')
h,w,c = image.shape

#Funciton to create custom kernel
def diamond_kernel(size):
    size = size if size%2 else size+1
    
    dkernel = numpy.zeros((size,size),dtype=numpy.uint8)
    center = size//2 + 1
    for i in range(size//2 +1):            #row idx
        for j in range(size):              #column idx
            dkernel[i][j] = j>=center-i-1 and j <=center+i-1
            dkernel[size-i-1][j] = j>=center-i-1 and j <=center+i-1

    return dkernel


# Set minimum and max HSV values
lower = numpy.array([20, 20, 0])
upper = numpy.array([32, 255, 255])

# Create HSV Image and threshold into a range.
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower, upper)

#Close the image with small kernel
mask = cv2.morphologyEx(mask,cv2.MORPH_CLOSE,kernel=diamond_kernel(7))

#Find the small blobs to filter out
contours,hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
    if cv2.contourArea(cnt) < 5000:
        cv2.fillPoly(mask,[cnt],(0,0,0))

#Close the image with big kernel
mask = cv2.morphologyEx(mask,cv2.MORPH_CLOSE,kernel=diamond_kernel(77))

#Get the masked image and find the max external contour which is the borderline
output = cv2.bitwise_and(image,image, mask= mask)

contours,hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
max_cnt = max(contours,key=cv2.contourArea) 
cv2.drawContours(image, [max_cnt],-1,(255,0,0),7)


#Show the results
cv2.imshow('High Kernel Closed Mask',cv2.resize(mask,(int(w*720/h),720)))
cv2.imshow('High Kernel Closed Output',cv2.resize(output,(int(w*720/h),720)))
cv2.imshow('Contours',cv2.resize(image,(int(w*720/h),720)))
cv2.waitKey()