I have a model with input that has a shape like (145, 24, 24, 3)
With model load by tensorflow keras output will have a shape like (145, 4)
But when I convert input from tensor to list and POST it into model serving.
Output return (,4)
I was using tensorflow-serving with docker
My code:
img_boxes = get_image_boxes(bboxes, img, height, width, num_boxes, size=24)
img_in = tf.make_tensor_proto(img_boxes)
img_in = tf.make_ndarray(img_in)
print(img_in.shape) # (145, 24, 24, 3)
# probs, offsets = self.rnet(img_boxes)
payload = {'instances': img_in.tolist()}
res = requests.post('http://localhost:8501/v1/models/r_net:predict', data=payload)
res = res.json()['predictions'][0]
probs = tf.convert_to_tensor(res['softmax'])
offsets = tf.convert_to_tensor(res['dense5_2'])
print(probs.shape) # (, 4)
And my model:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 24, 24, 3)] 0 []
permute (Permute) (None, 24, 24, 3) 0 ['input_1[0][0]']
conv1 (Conv2D) (None, 22, 22, 28) 784 ['permute[0][0]']
prelu1 (PReLU) (None, 22, 22, 28) 28 ['conv1[0][0]']
pool1 (MaxPooling2D) (None, 11, 11, 28) 0 ['prelu1[0][0]']
conv2 (Conv2D) (None, 9, 9, 48) 12144 ['pool1[0][0]']
prelu2 (PReLU) (None, 9, 9, 48) 48 ['conv2[0][0]']
pool2 (MaxPooling2D) (None, 4, 4, 48) 0 ['prelu2[0][0]']
conv3 (Conv2D) (None, 3, 3, 64) 12352 ['pool2[0][0]']
prelu3 (PReLU) (None, 3, 3, 64) 64 ['conv3[0][0]']
flatten (Flatten) (None, 576) 0 ['prelu3[0][0]']
dense4 (Dense) (None, 128) 73856 ['flatten[0][0]']
prelu4 (PReLU) (None, 128) 128 ['dense4[0][0]']
dense5_1 (Dense) (None, 2) 258 ['prelu4[0][0]']
softmax (Softmax) (None, 2) 0 ['dense5_1[0][0]']
dense5_2 (Dense) (None, 4) 516 ['prelu4[0][0]']
==================================================================================================
Total params: 100,178
Trainable params: 100,178
Non-trainable params: 0
__________________________________________________________________________________________________
My mistake was I did not get all predictions in output
res = res.json()['predictions'][0]
it should be:
res = res.json()['predictions']
, output is an array dictionary I need to concat them by the same keys I will get output as I want.
{'predictions': [{'softmax': [0.210848287, 0.789151728], 'dense5_2': [0.182407588, 0.159706563, 0.113356635, 0.401812047]}, {'softmax': [0.993124425, 0.0068756], 'dense5_2': [0.138127923, -0.0996344835, -0.020740509, -0.0633568913]}, {'softmax': [0.958264828, 0.0417351499], 'dense5_2': [0.0111884885, 0.0999818072, -0.130534053, 0.0480072]}, {'softmax': [0.983752847, 0.0162471626], 'dense5_2': [-0.0930862129, -0.224962771, 0.0652338713, 0.205662221]}, {'softmax': [0.947032034, 0.0529679693], 'dense5_2': [0.242067963, 0.0784763768, 0.0628610402, 0.109089941]}, {'softmax': [0.964857459, 0.0351425521], 'dense5_2': [0.210765839, 0.110972457, 0.0283354521, 0.210187465]},...]}