pythonopencvcomputer-visionobject-detectionsemantic-segmentation

How to detect (most) mortared stones with OpenCV findContours

I need to correctly outline as many as possible of the mortared stones in a street zone. The code below correctly detects some of them in the stones image "in.jpg", but it is not obvious why many remain undetected or only partly outlined. I'd also like to fix cases like contour 56, 57 or 92 by exploiting the fact that the stones are aligned and mostly oval image with detected contours "out.jpg". I know how to remove the obvious outliers, but before that I need to improve the detection method.

import cv2
img = cv2.imread('in.jpg')
gray = cv2.bitwise_not(img)
gray = cv2.cvtColor(gray, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (113, 113), 0)
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 61, 2)
thresh = cv2.dilate(thresh, None, iterations=5)
thresh = cv2.erode(thresh, None, iterations=1)
cnts, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
print(f'Found {len(cnts)} contours')
cv2.drawContours(img, cnts, -1, (0, 255, 0), 2)
centroids = [cv2.moments(c) for c in cnts]
for i, c in enumerate(centroids):
  cv2.putText(img, str(i), (int(c['m10'] / c['m00']), int(c['m01'] / c['m00']) - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
cv2.imwrite('out.jpg', img)

The various parameters (blur 113, etc.) have been obtained by trial and error.

It is not enough to just count the stones, I need to get their shape as closely as possible.

Thank you for your suggestions!

Contributing to Markus' question: here is an example of a mixed zone:

Here is another image from a mixed zone (partly mortared, partly not).

Solution

  • You can use cv2.moments(contour) for calculating the ratio between the contour length and the contour area. This will let you rule out non oval contours.

    Having said that, this is a relatively difficult problem for a classical CV approach. A neural network will do a better job (given enough training data) than findContours. This is mainly because the stones are quite different in their appearance and modeling them using a classical model requires many parameters that will capture this variance. Neural networks are naturally better suited for that kind of task.

    There are off the shelf segmenters like the SAM model that can do the job and a quick search shows there are also models designed specifically for stones segmentation.

    Here is an example of using SAM2 segmentation: enter image description here

    You can find a colab here