pythonopencvimage-processingcolor-detection

Color Detection not efficient on webcam images


I am working on a project where I'm trying to detect green and red circles on a specific surface (arena). When I try to do so with the digital version of that arena (PNG image), I can successfully detect both the colored circles.

Here's the digital image of the surface: Arena Image Original

Now, I printed this arena on a flex(without those two colored circles ), and manually placed coloured circular coins on it. But after capturing its image through a 1.3 MP webcam, the color detection didn't work and gave false results.

Here's the printed arena captured through webcam:

webcam image of printed arena

Why aren't the colors being detected? Do I need to do post-processing on webcam image? I've tried sharpening the image via cv2.filter2D but it didn't work either.

Here's a snippet for detecting Red circles from my Python code:

ip_image=cv2.imread("image.png")
kernel = np.array([[-1,-1,-1],[-1,9,-1],[-1,-1,-1]])
ip_image=cv2.filter2D(ip_image,-1,kernel)
#cv2.imshow("Hi",ip_image)
hsv=cv2.cvtColor(ip_image,cv2.COLOR_BGR2HSV)
red_low=np.array([0,255,255])
red_up=np.array([10,255,255])
mask0= cv2.inRange(hsv,red_low,red_up)
red_low=np.array([170,255,255])
red_up=np.array([180,255,255])
mask1=cv2.inRange(hsv,red_low,red_up)
mask_red=mask1+mask0
r_img= ip_image.copy()
r_img[np.where(mask_red==0)] = 0
gray_img0 = cv2.cvtColor(r_img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray_img0,0,255,0)
M = cv2.moments(thresh)
rX=int(M["m10"] / M["m00"])
rY=int(M["m01"] / M["m00"])
cv2.circle(ip_image,(rX,rY), 17, (255,255,255), 2)
cv2.imshow("Output",ip_image)

Also, the fourth parameter in cv2.threshold() when set to "0" works correctly with digital image while with webcam image it throws zero division error at the line :

rX=int(M["m10"]/M["m00"])

Solution

  • It is definitely worth trying to improve initial conditions (perspective, light, resolution, etc). Current result produced by the webcam is somewhat awful, so instead of spending lots of time fixing that it might be better to use less cheaper hardware.

    You can use some fancy methods to improve your image, but still, it is better to have more valuable input.

    enter image description here Anyway, here is the useful part. Your markers are not unique, so any attempt to use color will require additional shape analysis. Here are some results using color segmentation:

    enter image description here

    As you can see some areas have pretty similar colors. I use a bit more advanced color similarity function to handle complex cases. Basically, I specified red and green with some thresholds. Delta E will be the right starting point. Let's see the actual shapes:

    enter image description here

    With those results you can do a simple shape analysis or just compare areas to find your markers. I'd prefer to have a bit more unique colors and better initial conditions.

    Anyway, any real-life scene will require you to be really careful working with colors:

    enter image description here

    (see in action)

    Similar problem:

    Issue of the recognize people by their clothes color with not severe illumination environments