pythonopencvimage-processingimage-recognitionhaar-classifier

Stable icon recognition by opencv python

I have the following issue: I need to detect icons in an image reliably. The image also contains text, and the icons come in various sizes.

Currently, I'm using Python with the cv2 library for this task. However, unfortunately, the current contour detection algorithm using cv2.findContours isn't very reliable. Here's how I'm currently doing it:

gray = cv2.cvtColor(self.image, cv2.COLOR_BGR2GRAY)
binary = cv2.adaptiveThreshold(self.gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 17, 1)
contours, _ = cv2.findContours(self.binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Then follows contour filtering, merging filtered contours, and filtering again.

However, this method has proven to be unreliable.

I've also tried using getStructuringElement, but it gives unreliable results for icons on a white background.

I can't disclose real input data, but I used the Amazon logo and created an example demonstrating the issue.

enter image description here

For colored icons, when using contours, I often get two or three icons of incorrect sizes, and merging them loses the precise size. For icons on a white background, the approach with getStructuringElement doesn't detect the boundary well.

My questions: What do you suggest? My ideas:

  1. Train a Haar cascade. Would it help much?
  2. Fine-tune parameters of one of the current methods.
  3. Any other methods/libraries.

I'm open to any suggestions, or let me know if anyone has experience solving such a problem.

Img: https://i.sstatic.net/bZYrnCTU.jpg

PS: For another people, who's maybe be interested in more or less the same task

Answer from @Tino-d really great, Bot you may get and icon, which more or less would be divided into big amount of simple polygons

For example(After canny processing)

enter image description here

What you can do its just draw contours with bigger thickness(2 for example), and then refounding contours on new image

contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cv2.drawContours(edges, contours, -1, (255, 255, 255), 2)
contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

enter image description here

Seems like more or less super-easy and very fast merging contours algo)

Solution

  • I think edge detection can work quite well here, I did a small example which worked for now:

    im = cv2.imread("logos.jpg")
imGray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(imGray,5, 20)

    This gave the following result:

    edges

    After this, detecting contours and filtering by area will work quite nicely, as the squares of the logos seem to all be the same size:

    contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
sortedContours = sorted(contours, key = cv2.contourArea, reverse = True)
for c in sortedContours:
    print(cv2.contourArea(c))

    We see that indeed the three biggest contours by area all have around 10500 pixels:

    10787.0
10715.0
10128.0
7391.5
4555.5
3539.0
3420.0
.
.
.

    Fill those first three contours:

    im1 = cv2.drawContours(im.copy(), sortedContours, 2, (255,0,0), -1)
im2 = cv2.drawContours(im.copy(), sortedContours, 1, (0,255,0), -1)
im3 = cv2.drawContours(im.copy(), sortedContours, 0, (0,0,255), -1)

    And this is what you will get:

    filled logos

    I assume what you want is a boolean mask to be able to get those pixels. So something like

    mask = np.zeros_like(imGray)
mask = cv2.drawContours(mask, sortedContours, 2, 1, -1)
firstLogo = cv2.bitwise_and(im, im, mask = mask)

    Can do the job. You can automate this quite easily by filtering the contours, I'm just nudging a POC to you.

    E: forgive the weird colours, forgot to convert to RGB before imshow with Matplotlib...