tensorflowtensorflow.jsmozilla-deepspeech

Converting Mozilla DeepSpeech model to use in tensorflow.js environment

I have been attempting to convert a Mozilla Deepspeech trained model for use in tensorflow.js, in ml5.js soundClassifier layer. My understanding is that Mozilla DeepSpeech uses TensorFlow. I have been trying to follow the hints found here:

https://www.tensorflow.org/js/tutorials/conversion/import_saved_model

https://www.tensorflow.org/js/guide/conversion

tensorflowjs_converter --help

I have downloaded the DeepSpeech model from here:

https://github.com/mozilla/DeepSpeech/releases/download/v0.6.1/deepspeech-0.6.1-models.tar.gz

and after unpacking, the following files are found:

lm.binary output_graph.pb output_graph.pbmm output_graph.tflite trie

I have attempted to run commands such as:

tensorflowjs_converter --output_format=tfjs_graph_model --saved_model_tags=serve deep/ tensorflow.js/

and variations to do the conversion. tensorflow.js/ is a directory I created, deep/ is the directory containing the DeepSpeech model files (which are listed above.)

I get error:

SavedModel file does not exist at: deep/saved_model.pb/{saved_model.pbtxt|saved_model.pb}

I renamed, eg, output_graph.pb to saved_model.pb.

Firstly, I would like to know if the DeepSpeech model is even compatible with the tensorflowjs_converter, and if so, what am I missing to get this thing working.

Solution

  • i believe the output_graph.pb is a frozen model. in which case, you would need to run the converter like such:

    tensorflowjs_converter \
    --input_format=tf_frozen_model \
    --output_node_names='logits' \
    deepspeech/output_graph.pb \
    tensorflowjs

    however, doing this results in the following error:

    ValueError: Unsupported Ops in the model before optimization
BlockLSTM, Mfcc, AudioSpectrogram

    the operations BlockLSTM, Mfcc, and AudioSpectrogram are currently not supported by TensorFlow.js so the conversion will fail: