I'm trying to serve a TensorFlow model (using TensorFlow Serving). It seems that in order to use the TensorFlow Serving APIs and make predictions, I need to use the make_tensor_proto method.
For my Docker image, I'm installing 1.3GB of the TensorFlow package just for that one method. Obviously, this is not ideal. So, I'm wondering if I can import make_tensor_proto
from a different (leaner) package or use an alternative method.
You can follow the steps mentioned in the link below.
https://www.mux.com/blog/tuning-performance-of-tensorflow-serving-pipeline
Simplified steps:
From this line of code
import tensorflow as tf
...
tensor = tf.contrib.util.make_tensor_proto(features)
request.inputs['inputs'].CopyFrom(tensor)
To this line of code
from protos.tensorflow.core.framework import tensor_pb2
from protos.tensorflow.core.framework import tensor_shape_pb2
from protos.tensorflow.core.framework import types_pb2
...
# ensure NHWC shape and build tensor proto
tensor_shape = [1]+list(img.shape)
dims = [tensor_shape_pb2.TensorShapeProto.Dim(size=dim) for dim in tensor_shape]
tensor_shape = tensor_shape_pb2.TensorShapeProto(dim=dims)
tensor = tensor_pb2.TensorProto(
dtype=types_pb2.DT_FLOAT,
tensor_shape=tensor_shape,
float_val=list(img.reshape(-1)))
request.inputs['inputs'].CopyFrom(tensor)
You can copy this compiled pythonic functions into your projects working directory and copy into the docker which should work.