NodeDef mentions attr \'batch_dims\' not in Op in tensorflow serving

I'm writing a ner model using keras, and deploying models to tensorflow-serving. Then the http requests is used to get the prediction results.

Here are my codes:

EMBEDDING_OUT_DIM = 128
TIME_STAMPS = 100
HIDDEN_UNITS = 200
DROPOUT_RATE = 0.3

def crf_model(VOCAB_SIZE, TAGS_NUMS):
    model = Sequential()
    model.add(Embedding(VOCAB_SIZE, output_dim=EMBEDDING_OUT_DIM, mask_zero=True, input_length=TIME_STAMPS))
    model.add(Bidirectional(LSTM(HIDDEN_UNITS, return_sequences=True)))
    model.add(Dropout(DROPOUT_RATE))
    model.add(Bidirectional(LSTM(HIDDEN_UNITS, return_sequences=True)))
    model.add(Dropout(DROPOUT_RATE))
    model.add(TimeDistributed(Dense(TAGS_NUMS)))
    crf = CRF(TAGS_NUMS, sparse_target=True)
    model.add(crf)
    model.summary()
    model.compile('rmsprop', loss=crf.loss_function, metrics=[crf.accuracy])
    return model

def save_model_to_serving(model, export_version, export_path='ner_serving'):
    print(model.input, model.output)
    signature = tf.saved_model.signature_def_utils.predict_signature_def( 
        inputs={'input_ids': model.input}, outputs={'outputs': model.output})
    export_path = os.path.join(
        tf.compat.as_bytes(export_path),
        tf.compat.as_bytes(str(export_version)))
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
    builder.add_meta_graph_and_variables(
        sess=K.get_session(),                                                                                                                    
        tags=[tf.saved_model.tag_constants.SERVING],                                                                                             
        signature_def_map={                                                                                                                      
            'ner': signature,                                                                                                                     
        },
        legacy_init_op=legacy_init_op)
    builder.save()

The http request is as follows:

test_data = dict()
test_data['inputs'] = [0] * 100


data = json.dumps({"signature_name": "ner", "instances": [test_data]})
print(data)

headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:9001/v1/models/ner-serving:predict', data=data, headers=headers)

print(json_response.text)

predictions = json.loads(json_response.text)

print(predictions)

The tensorflow-serving returns NodeDef mentions attr \'batch_dims\' not in Op

Make sure the tensorflow version you used to do the training and model saving is the same as the tensorflow/serving container you used for serving.

I had the same error message when I used tensorflow 1.14 for training but an older container image of tensorflow/serving v1.13 for serving. Once I used v1.14 for both stages, it worked.