pythonopencvtrackingbounding-box

Check to see if a bounding box of an object is inside the bounding box of another object


I have a video with people walking around different homes. I created a fixed bounding box around the different homes and implemented a tracker to track the movements of the people. I now want to check if the bounding box of any of the people are inside the bounding boxes of any of the homes.

Since this is a video, the check of True or False - i.e. whether a person inside a bounding box or not would constantly change as the people walk around. As an example, I have included a picture of the starting frame of the video. As you can see, Person 0 is not inside the bounding box of any of the houses at all. However, it prints True anyway. Also, I'd also just like to say that I checked multiple other sources and posts on this platform to seek some guidance but I couldn't find anything that could assist me with my problem.

Output that I'm getting:

Person 0 : True
Person 1 : True
Person 2 : True
Person 0 : True
Person 1 : True
...

Expected output would be something as follows (first 3 lines of the expected output correspond with the image below):

Person 0: False
Person 1: True
Person 2: True
Person 0: False
Person 1: True
...

enter image description here

This is what I have done so far:

# boxes_houses = [(#, #, #, #), (#, #, #, #), ...]

while cap.isOpened():
    success, frame = cap.read()
    if not success:
        break

    # get updated location of objects in subsequent frames
    success, boxes_person = multiTracker.update(frame)
    # boxes_persons = [(#, #, #, #), (#, #, #, #), ...]

    cnt_person = 0
    cnt_house = 0

    for box_person, box_house in zip(boxes_persons, boxes_houses):
        x_p1 = int(box_person[0])
        y_p1 = int(box_person[1])
        x_p2 = int(box_person[2])
        y_p2 = int(box_person[3])
        person_coords = {'x1': x_p1, 'y1': y_p1, 'x2': x_p2, 'y2': y_p2}
        cv2.rectangle(frame, (x_p1, y_p1), (x_p1 + x_p2, y_p1 + y_p2), (0, 0, 0), 2, 1)
        cv2.putText(frame, "House: {}".format(cnt_house), (x1, y1 - 10), cv2.FONT_HERSHEY_COMPLEX_SMALL, 1, (0, 0, 0), 1)

        x_h1 = int(box_house[0])
        y_h1 = int(box_house[1])
        x_h2 = int(box_house[2])
        y_h2 = int(box_house[3])
        cv2.rectangle(frame, (x_h1 , y_h1), (x_h1 + x_h2, y_h1+ y_h2), (0, 0, 255), 2, 1)
        cv2.putText(frame, "Person: {}".format(cnt_person ), (int(box_house[0]), int(box_house[1] - 5)), cv2.FONT_HERSHEY_COMPLEX_SMALL, 1, (0, 0, 255), 1)

        ## CHECK TO SEE IF BBOX OF PERSONS ARE INSIDE BBOX OF HOMES ##

        house1 = {'x1': #, 'y1': #, 'x2': #, 'y2': #}
        house2 = {'x1': #, 'y1': #, 'x2': #, 'y2': #}
        house3 = {'x1': #, 'y1': #, 'x2': #, 'y2': #}

        if (person_coords['x1'] < house1['x1'] and person_coords['y1'] < house1['y1'] or person_coords['x2'] < house1['x2'] and person_coords['y2'] < house1['y2']) or (person_coords['x1'] < house2['x1'] and person_coords['y1'] < house2['y1'] or person_coords['x2'] < house2['x2'] and person_coords['y2'] < house2['y2']) or (person_coords['x1'] < house3['x1'] and person_coords['y1'] < house3['y1'] or person_coords['x2'] < house3['x2'] and person_coords['y2'] < house3['y2']):
            print ("Person", num_person, ": True\n")
        
        else:
            print ("Person", num_person, ": False\n")
        
        cnt_house+= 1
        cnt_person+= 1

    # show frame
    cv2.imshow('MultiTracker', frame)

    # quit on ESC button
    if cv2.waitKey(1) & 0xFF == 27:  # Esc pressed
        break

Any help on this would be much appreciated!


Solution

  • I think the issue comes from the position check (which is very hard to read since it's all on one line in an if statement).

    Let's first define a function to check whether a person is in a given house:

    def isInside(person, house):
      """
        person is the dict with x1, y1, x2 and y2 for that person
        house is the dict with x1, y1, x2 and y2 for that house
      """
      
      # First we make sure we compare things in the right order
      # You can skip that part if you are sure that in all cases x1 < x2 and y1 < y2
      p_xmin = min(person["x1"], person["x2"])
      p_xmax = max(person["x1"], person["x2"])
      p_ymin = min(person["y1"], person["y2"])
      p_ymax = max(person["y1"], person["y2"])
      
      h_xmin = min(house["x1"], house["x2"])
      h_xmax = max(house["x1"], house["x2"])
      h_ymin = min(house["y1"], house["y2"])
      h_ymax = max(house["y1"], house["y2"])
      
      # Then you perform your checks. From what I understood,
      # you want the result to be true if any corner of the person
      # is inside the house's bounding box.
      
      p_corners = [
        (p_xmin, p_ymin),
        (p_xmin, p_ymax),
        (p_xmax, p_ymin),
        (p_xmax, p_ymax)]
    
      for corner in p_corners:
        in_range_along_x = corner[0] < h_xmax and corner[0] > h_xmin
        in_range_along_y = corner[1] < h_ymax and corner[1] > h_ymin
        if in_range_along_x and in_range_along_y:
          return True
      
      # If we get there, then the person is not inside that house
      return False
    

    You can also modify the function above to use the center of the person's bounding box instead of its corners, or to check that all corners are inside. Here is the center version as a bonus:

    def centerIsInside(person, house):
      """
        person is the dict with x1, y1, x2 and y2 for that person
        house is the dict with x1, y1, x2 and y2 for that house
      """
      
      # First we make sure we compare things in the right order
      # You can skip that part if you are sure that in all cases x1 < x2 and y1 < y2
    
      h_xmin = min(house["x1"], house["x2"])
      h_xmax = max(house["x1"], house["x2"])
      h_ymin = min(house["y1"], house["y2"])
      h_ymax = max(house["y1"], house["y2"])
    
      # We compute the center of the person:
    
      p_xcenter = (person["x1"]+person["x2"])/2
      p_ycenter = (person["y1"]+person["y2"])/2
    
      # Then you perform your checks.
    
      in_range_along_x = p_xcenter < h_xmax and h_xmin > p_xcenter
      in_range_along_y = p_ycenter < h_ymax and h_ymin > p_ycenter
      
      return in_range_along_x and in_range_along_y
    

    You can then use that function to perform the checks instead of that long if statement:

    # [...]
    
    houses = [house1, house2, house3]
    isinside_checks = []
    for house in houses:
      isinside_checks.append(isInside(person_coords, house))
    if any(inside_checks):
      print ("Person", num_person, ": True\n")
    else:
      print ("Person", num_person, ": False\n")
    
    # [...]
    

    You can also modify the isInside function to make it perform different types of checks depending on an argument (any corner, center, all corners...)