pythonopencvvideoobject-detectionobject-tracking

OpenCV-Python overlapping boundingRect()


I'm trying to fix a bug where when two rects collide in a video, they merge into one big rectangle for the remaining frames that they collide, instead of keeping the rectangles as before:

Before collision:

After collision:

I want to keep both of the rectangles even when the objects collide.

Main loop snippet:

while True:
    ret, frame = vid.read()
    if frame is None:
        break

    # TRESH_OTSU has problems with objects that are very dark
    bright_frame = cv2.addWeighted(
        frame, 4.3, np.zeros(frame.shape, frame.dtype), 0, 20
    )

    # create a mask that will remove the black background from tracking
    gray = cv2.cvtColor(bright_frame, cv2.COLOR_BGR2GRAY)
    _, mask = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

    # get the moving objects
    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    detections = []
    for obj in contours:
        area = cv2.contourArea(obj)
        if area > 50:  # filter noise
            perimeter = cv2.arcLength(obj, True)
            approx = cv2.approxPolyDP(obj, 0.02 * perimeter, True)
            object_type = "rectangle" if len(approx) == 4 else "circle"
            x, y, w, h = cv2.boundingRect(obj)
            detections.append(
                DetectedObject(
                    x, y, w, h, get_obj_color(frame, x, y, w, h), object_type
                )
            )

    rects = tracker.update(detections)
    for rect in rects:
        x, y, w, h, id, _, _, time = rect
        cv2.rectangle(frame, (x, y), (x + w, y + h), (255, 255, 255), 5)
        # cv2.putText(
        #     frame, str(id), (x, y - 15), cv2.FONT_HERSHEY_PLAIN, 5, (255, 255, 255)
        # )
        print(rect)
    cv2.imshow("frame", frame)

    key = cv2.waitKey(10)

    # break the loop upon the press of esc
    if key == 27:
        break

Solution

  • Here's an illustration of what I mean. I did a small example because you did not supply an unprocessed image:

    Separated:

    sep

    Colliding:

    behind

    The approach is pretty simple:

    1. define boundaries
    2. apply cv2.inRange
    3. get positive pixels and get bounding rectangle

    For the separated:

    sepWithBox

    For colliding:

    behind

    Code snippet:

    # define boundary for first object
    lower1 = (150,90,30)
    upper1 = (170,110,60)
    # define boundary for second object
    lower2 = (150,150,150)
    upper2 = (255,255,255)
    # get bounding boxes
    boundingRect1 = cv2.boundingRect(cv2.inRange(im,lower1,upper1))
    boundingRect2 = cv2.boundingRect(cv2.inRange(im,lower2,upper2))
    # draw rectangle on im
    cv2.rectangle(im, boundingRect1, (255,0,0), 5)
    cv2.rectangle(im, boundingRect2, (0,0,255), 5)
    

    Am not sure how many objects you will eventually have, nor am I sure that the two objects can be necessarily separated colour-wise. Eventually look at connected components if you always have clean object like photos