pythonnumpyopencvimage-processingmediapipe

How to detect red spots (pimples) on skin using OpenCV


I have just started out into exploring OpenCV and MediaPipe and I have to detect and mark pimples on faces for a task (P.S. I have no background into image processing).

However, I cannot yet differentiate between pimples and spots, especially so for dark skin faces. My current code also marks every red area of the face including lips, etc.

Can anyone please help me or give me any pointers on what I could do? I am trying out using mediapipe face mesh to maybe crop out a certain area of the face but it hasnt worked well yet. I really have no idea except for maybe marking contours onto the skin (which doesnt work and marks things other than acne). detect pimples on clear face for check, pimple detection, not so accurate pimple detection Currently, I have the following code:

import cv2
import numpy as np

image = cv2.imread('./acne.jpeg')
cv2.imshow("Original", image)

result = image.copy()

image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

# lower boundary RED color range values; Hue (0 - 10)
lower1 = np.array([0, 115, 20])
upper1 = np.array([10, 255, 255])

# upper boundary RED color range values; Hue (160 - 180)
lower2 = np.array([160, 105, 20])
upper2 = np.array([179, 255, 255])

lower_mask = cv2.inRange(image, lower1, upper1)
upper_mask = cv2.inRange(image, lower2, upper2)

full_mask = lower_mask + upper_mask

result = cv2.bitwise_and(result, result, mask=full_mask)
edges = cv2.Canny(image=full_mask, threshold1=0, threshold2=100)

cv2.imshow('edges', edges)

cv2.imshow('mask', full_mask)

# cv2.imshow('result', result)

cv2.waitKey(0)
cv2.destroyAllWindows()


Solution

  • Here is my suggestion. You may apply the following steps by one by and I believe you will get the pimples at the end.

    1. Actually you are able to get the pimples via on canny result but the problem you are meeting is to get rid of hair, nose, eyes etc. contours from the result. If we delete those parts from the result, you will only get the pimples at the end.
    2. Since you are familiar with mediapipe, I will suggest you to get the landmarks of each face with mediapipe. Mediapipe is faster, accurate and also compatible with python. Here is a starting example of face landmark detection.
    3. After getting the landmarks of each face, you can easily get rid of hair by using convexHull of opencv.
    4. Inside the landmark points area, there are still eyes, eyebrows, lips etc. Mediapipe also gives you the positions of these face parts. So in a similar way with hair removing, you can also remove(ignore) these parts.
    5. At the end, you cansearch your pimples in the clean parts of the face.