pythonclasstensorfaster-rcnndetectron

Replace instance of a class with new instance

I am working on detectron2 object detection. I am facing a problem in filtering the objects detected.

Here is the detectron2 predicted output:

Instances(num_instances=9, image_height=547, image_width=820, fields=[pred_boxes: Boxes(tensor([[3.1173e+01, 3.8368e+01, 5.3751e+02, 5.4078e+02],
    [5.9945e+02, 2.6412e+02, 6.8196e+02, 5.1333e+02],
    [4.4486e+02, 1.7210e+02, 4.9981e+02, 2.5596e+02],
    [1.1566e-01, 2.3533e+02, 8.5483e+01, 3.6838e+02],
    [3.0897e+02, 2.4964e+02, 3.5739e+02, 4.8948e+02],
    [7.6962e-03, 2.3240e+02, 8.5447e+01, 3.7128e+02],
    [2.7454e+02, 2.6212e+02, 3.3122e+02, 4.5928e+02],
    [6.4399e+02, 3.0057e+02, 6.6374e+02, 3.8033e+02],
    [3.1025e+02, 2.5372e+02, 3.3572e+02, 3.5059e+02]])), scores: tensor([0.9998, 0.9994, 0.9941, 0.8815, 0.8447, 0.3559, 0.1484, 0.1304, 0.0928]), pred_classes: tensor([ 0,  0, 67,  2, 27,  7, 27, 27, 27]), pred_masks: tensor([[[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    ...,

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]],

    [[False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     ...,
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False],
     [False, False, False,  ..., False, False, False]]])])

I did the filtering and created a new list(dict) with predicted object classes, scores and boxes. I wanted to plot and visualize this on image:

Filtering code:

idxofClass = [i for i, x in enumerate(list(outputs['instances'].pred_classes)) if (x == 0)]
outputs_new = [{'pred_classes': o.pred_classes[idxofClass], 'scores':o.scores[idxofClass], 'pred_boxes':o.pred_boxes[idxofClass] }]

Now, I able to get the filtered values as below:

[{'pred_classes': tensor([ 0,  0, 67]), 'scores': tensor([0.9998, 0.9994, 0.9941]), 'pred_boxes': Boxes(tensor([[ 31.1728,  38.3685, 537.5092, 540.7788],
    [599.4498, 264.1228, 681.9622, 513.3326],
    [444.8603, 172.1017, 499.8055, 255.9632]]))}]

While passing this value to Visualizer, getting the below error:

Traceback (most recent call last):
  File "apimodel.py", line 96, in <module>
    out = v.draw_instance_predictions(outputs_new)
  File "/root/anaconda3/envs/ml-engine/lib/python3.8/site-packages/detectron2/utils/visualizer.py", line 366, in draw_instance_predictions
    boxes = predictions.pred_boxes if predictions.has("pred_boxes") else None
AttributeError: 'list' object has no attribute 'has'

The data type of original output is a class instance:

o = outputs["instances"]
print("data type:", type(o))
<class 'detectron2.structures.instances.Instances'>

The output of newly created filtered output is a list(dict):

<class 'list'>

My objective is to plot the bounding box based on filtered score. I have been trying to replace original values of output, but not successful. Please assist on this.

Solution

  • After two days of searching, I found a way to achieve my objective. I am writing answer, as detectron2 class, so that, if anyone looking for similar approach will get benefit.

    Filter Index of classes:

    idxofClass = [i for i, x in enumerate(list(outputs['instances'].pred_classes)) if x == 0]

    Create new class, boxes, scores & masks:

    classes = o.pred_classes[idxofClass]
scores = o.scores[idxofClass]
boxes = o.pred_boxes[idxofClass]
masks = o.pred_masks[idxofClass]

    Define new instance and set the new values to new instance. Note: detectron2 module provides this method set.

    obj = detectron2.structures.Instances(image_size=(480, 640))

obj.set('pred_classes', classes)
obj.set('scores', scores)
obj.set('pred_boxes', boxes)
obj.set('pred_masks', masks)

    Now you can use this new instance obj for other processing and visualization:

    out = v.draw_instance_predictions(obj.to("cpu"))