pythontensorflowtensorflow-serving

Is there a way to use make_tensor_proto without having to install the entire TensorFlow package?


I'm trying to serve a TensorFlow model (using TensorFlow Serving). It seems that in order to use the TensorFlow Serving APIs and make predictions, I need to use the make_tensor_proto method.

For my Docker image, I'm installing 1.3GB of the TensorFlow package just for that one method. Obviously, this is not ideal. So, I'm wondering if I can import make_tensor_proto from a different (leaner) package or use an alternative method.


Solution

  • You can follow the steps mentioned in the link below.

    https://www.mux.com/blog/tuning-performance-of-tensorflow-serving-pipeline

    Simplified steps:

    1. Copy the tensor protos to the local machine.
    2. Compile the protos to python module and place it under the project directory
    3. Replace tensorflow library with proto pythonic functions.

    From this line of code

    import tensorflow as tf  
    ...
    tensor = tf.contrib.util.make_tensor_proto(features)  
    request.inputs['inputs'].CopyFrom(tensor)
    

    To this line of code

    from protos.tensorflow.core.framework import tensor_pb2  
    from protos.tensorflow.core.framework import tensor_shape_pb2  
    from protos.tensorflow.core.framework import types_pb2  
    ...
    # ensure NHWC shape and build tensor proto
    tensor_shape = [1]+list(img.shape)  
    dims = [tensor_shape_pb2.TensorShapeProto.Dim(size=dim) for dim in tensor_shape]  
    tensor_shape = tensor_shape_pb2.TensorShapeProto(dim=dims)  
    tensor = tensor_pb2.TensorProto(  
                  dtype=types_pb2.DT_FLOAT,
                  tensor_shape=tensor_shape,
                  float_val=list(img.reshape(-1)))
    request.inputs['inputs'].CopyFrom(tensor) 
    

    You can copy this compiled pythonic functions into your projects working directory and copy into the docker which should work.