I am following this link to train an object detection model. I am able to successfully deploy the model on EC2 instance. The accuracy was good. I complied the same model file for m edge Device Jetson Nano. My inference code looks like below,
from dlr import DLRModel
import json
import cv2
model = DLRModel('model', 'gpu')
img = cv2.imread('test.jpg')
img = cv2.resize(img, (512, 512), interpolation=cv2.INTER_AREA)
img = np.expand_dims(img, 0)
outputs = model.run(img)
objects=outputs[0][0]
scores=outputs[1][0]
bounding_boxes=outputs[2][0]
When I look the result, it's not at all matched with SageMaker Notebook instance result. Boudning boxes' values are sometime in ~70000. I couldn't understand the format of result produced by DLR.
Sample result for an image.
Classes: [[10.0], [14.0], [4.0], [10.0], [14.0], [-1.0], [-1.0], [-1.0], [7.0], [-1.0], [6.0], [11.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [2.0], [-1.0], [-1.0], [0.0], [-1.0], [17.0], [-1.0], [-1.0], [6.0], [18.0], [-1.0], [-1.0], [-1.0], [18.0], [-1.0], [12.0], [-1.0], [-1.0], [13.0], [-1.0], [-1.0], [-1.0], [1.0], [-1.0], [-1.0], [5.0], [-1.0], [0.0], [-1.0], [-1.0], [-1.0], [-1.0], [3.0], [8.0], [5.0], [-1.0], [-1.0], [15.0], [-1.0], [9.0], [3.0], [-1.0], [10.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [16.0], [8.0], [-1.0], [16.0], [19.0], [-1.0], [9.0], [-1.0], [4.0], [-1.0], [-1.0], [15.0], [10.0], [-1.0], [-1.0], [4.0], [-1.0], [8.0], [2.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0]]
scores: [[0.9527158737182617], [0.910746157169342], [0.28013306856155396], [0.059000786393880844], [0.04898739233613014], [-1.0], [-1.0], [-1.0], [0.04864144325256348], [-1.0], [0.04847110062837601], [0.04843416064977646], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [0.04821664094924927], [-1.0], [-1.0], [0.04808368161320686], [-1.0], [0.0479729063808918], [-1.0], [-1.0], [0.04782549664378166], [0.04778601974248886], [-1.0], [-1.0], [-1.0], [0.0475776381790638], [-1.0], [0.047538403421640396], [-1.0], [-1.0], [0.047468967735767365], [-1.0], [-1.0], [-1.0], [0.04737424850463867], [-1.0], [-1.0], [0.047330085188150406], [-1.0], [0.04730956256389618], [-1.0], [-1.0], [-1.0], [-1.0], [0.04710235074162483], [0.04710135608911514], [0.047083333134651184], [-1.0], [-1.0], [0.047033149749040604], [-1.0], [0.0469636432826519], [0.046939317137002945], [-1.0], [0.04687687009572983], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [0.046708278357982635], [0.046680934727191925], [-1.0], [0.04660974070429802], [0.046597886830568314], [-1.0], [0.04656397923827171], [-1.0], [0.046513814479112625], [-1.0], [-1.0], [0.04647510126233101], [0.04644943028688431], [-1.0], [-1.0], [0.046245038509368896], [-1.0], [0.01647786796092987], [0.010514501482248306], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0], [-1.0]]
Bouding boxes: [[523.2319946289062, -777751.875, 523.5722045898438, 778992.625], [425.0546875, -1141093.0, 429.6099853515625, 1142524.5], [680.2073364257812, 542.9566650390625, 680.2202758789062, 543.36376953125], [425.0546875, -1141093.0, 429.6099853515625, 1142524.5], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [94.88882446289062, -59.820770263671875, 417.0069885253906, 571.7384033203125], [21.017044067382812, 20.89019775390625, 491.50994873046875, 491.19842529296875], [30.993896484375, 30.63165283203125, 480.84320068359375, 481.7999267578125], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [-60.170318603515625, 95.33354187011719, 571.584716796875, 417.34515380859375], [591.7652587890625, -1121890.0, 592.9493408203125, 1123299.75], [523.2319946289062, -777751.875, 523.5722045898438, 778992.625], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0], [-1.0, -1.0, -1.0, -1.0]]
What causes this notorious result? Is there any issue while compiling model in Neo? Any issue in inference Code?
Any hint would be appreciable.
The reason behind the abnormal result is due to improper pre-processing method was applied. Here is the complete inference code for Mobilenet-ssd model.
def transform(img):
# Normalize
mean_vec = np.array([0.485, 0.456, 0.406])
stddev_vec = np.array([0.229, 0.224, 0.225])
image = (img / 255 - mean_vec) / stddev_vec
# Transpose
if len(image.shape) == 2: # for greyscale image
image = np.expand_dims(image, axis=2)
image = np.rollaxis(image, axis=2, start=0)[np.newaxis, :]
return image
model = DLRModel(model_dir, 'gpu')
for file_name in image_folder:
image = PIL.Image.open(file_name)
image = np.asarray(image.resize((512, 512)))
image = transform(image)
# flatten within a input array
input_data = {'data': image}
outputs = model.run(input_data)
objects = outputs[0]
scores = outputs[1]
bounding_boxes = outputs[2]
result = [objects.tolist(), scores.tolist(), bounding_boxes.tolist()]
print(json.dumps(result))