How to know if I need to reverse the thresholding TYPE after findContour

I'm working with OpenCV on hand detection. But I'm struggling when trying to contours of the threshed image. findContour will always try to find white area as contour.

So basically it works in most cases but sometimes my threshed image looks like this :

_, threshed = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY|cv2.THRESH_OTSU)

So to make it works I just need to change the threshold type cv2.THRESH_BINARY_INV.

_, threshed = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV|cv2.THRESH_OTSU)

And it works well.

My question is how can I determine when the threshold need to be reversed ? Does I Need to always found contours on both threshed images, and compare the result (I this case How ?) ? or there is a way to allays knows if contours are not totally missed.

EDIT : There is a way to be 100% sure contour looks like a hand ?

EDIT 2 : So I forgot to mention that I'm trying to detect fingertips and defects using this method so I need defects, which with the first threshed image I can't find them, because it reversed. See blue point on the First contour image.

Thanks.

You can write a utility method to detect the most dominant color along the border and then decide the logic, as if you want to invert the image or not, so the flow may look like:

1. Use OSTU binarization method.
2. Pass the thresholded image to utility method get_most_dominant_border_color and get the dominant color.
3. If the border color is WHITE, then you should invert the image using cv2.bitwise_not, else keep it that way only.

The get_most_dominant_border_color could be defined as:

from collections import Counter

def get_most_dominant_border_color(img):
    # Get the top row
    row_1 = img[0, :]
    # Get the left-most column
    col_1 = img[:, 0]
    # Get the bottom row
    row_2 = img[-1, :]
    # Get the right-most column
    col_2 = img[:, -1]

    combined_li = row_1.tolist() + row_2.tolist() + col_1.tolist() + col_2.tolist()

    color_counter = Counter(combined_li)

    return max(color_counter.keys(), key=lambda x:color_counter.values())