How can I solve OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'rectangle'?

I study on a instance segmentation model. I complede training part. BUt in the inference part; i faced with an error as I showed below. I converted boxes[i][0] to the int(boxes[i][0]) but it gives also error. Are there any idea to help?

def get_prediction(img_path, confidence):
    
    img = Image.open(img_path)
    transform = transforms.ToTensor()
    img = transform(img)
    img = img.to(device)
    pred = model([img])
    pred_score = list(pred[0]['scores'].detach().cpu().numpy())
    pred_t = [pred_score.index(x) for x in pred_score if x>confidence][-1]
    masks = (pred[0]['masks']>0.5).squeeze().detach().cpu().numpy()
    # print(pred[0]['labels'].numpy().max())
    pred_class = [CLASS_NAMES[i] for i in list(pred[0]['labels'].cpu().numpy())]
    pred_boxes = [[(i[0], i[1]), (i[2], i[3])] for i in list(pred[0]['boxes'].detach().cpu().numpy())]
    masks = masks[:pred_t+1]
    pred_boxes = pred_boxes[:pred_t+1]
    pred_class = pred_class[:pred_t+1]
    return masks, pred_boxes, pred_class

def segment_instance(img_path, confidence=0.5, rect_th=2, text_size=2, text_th=2):

    masks, boxes, pred_cls = get_prediction(img_path, confidence)
    img = cv2.imread(img_path)
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    for i in range(len(masks)):
      rgb_mask = get_coloured_mask(masks[i])
      img = cv2.addWeighted(img, 1, rgb_mask, 0.5, 0)
      cv2.rectangle(img, boxes[i][0], boxes[i][1],color=(0, 255, 0), thickness=rect_th)
      cv2.putText(img,pred_cls[i], boxes[i][0], cv2.FONT_HERSHEY_SIMPLEX, text_size, (0,255,0),thickness=text_th)
    plt.figure(figsize=(20,30))
    plt.imshow(img)
    plt.xticks([])
    plt.yticks([])
    plt.show()



segment_instance('image1.png', confidence=0.4)

error:

Input In [31], in segment_instance(img_path, confidence, rect_th, text_size, text_th)
     20   rgb_mask = get_coloured_mask(masks[i])
     21   img = cv2.addWeighted(img, 1, rgb_mask, 0.5, 0)
---> 22   cv2.rectangle(img, boxes[i][0], boxes[i][1],color=(0, 255, 0), thickness=rect_th)
     23   cv2.putText(img,pred_cls[i], boxes[i][0], cv2.FONT_HERSHEY_SIMPLEX, text_size, (0,255,0),thickness=text_th)
     24 plt.figure(figsize=(20,30))

error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'rectangle'

Overload resolution failed:

Can't parse 'pt1'. Sequence item with index 0 has a wrong type

Can't parse 'pt1'. Sequence item with index 0 has a wrong type

argument for rectangle() given by name ('color') and position (3)

argument for rectangle() given by name ('color') and position (3)

Solution

cv::rectangle has two overloads, where the first takes two cv::Points and the second takes a cv::Rect. Both are tuples in Python, where the elements are integers.

Creating a tuple for pt argument like the following may work.

pt1 = tuple([int(j) for j in boxes[i][0]])
pt2 = tuple([int(j) for j in boxes[i][1]])
cv2.rectangle(img, pt1, pt2, color=(0, 255, 0), thickness=rect_th)

As for the second overload, note that cv::Rect consists of (x, y, width, height).

pred_boxes = [[(i[0], i[1]), (i[2], i[3])] for i in list(pred[0]['boxes'].detach().cpu().numpy())]

Given that you create boxes yourself, you can make the coordinates integer in the first place.

pred_boxes = [[(i[0], i[1]), (i[2], i[3])] for i in list(pred[0]['boxes'].detach().cpu().numpy().astype(int))]