tensorflowkerasspeech-recognitiontensorflow.jstensorflowjs-converter

Convert Subclassed Speech Recognition Model to Tensorflow.js


I have a subclassed Speech Recognition model (link) with which I'd like to make inferences on my node.js server. I am trying to convert it using tfjs but because its a subclassed model I'm getting the following error:

NotImplementedError: Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider saving to the Tensorflow SavedModel format (by setting save_format="tf") or using `save_weights`.

I am following the official tutorial, which doesn't count this scenario in. And surprisingly I couldn't find much info on the web apart from a closed issue.

Any ideas on how to convert a Subclassed Model to tensorflowjs?


Solution

  • Ok, so I was specifically trying to convert a speech recognition model (link above) and it seems that most such models aren't supported at the moment by tfjs (including mozilla's deepspeech).

    It will specifically throw this error:

    ValueError: Unsupported Ops in the model before optimization
    AudioSpectrogram
    

    The command used being in this case:

    tensorflowjs_converter path/to/qnet15/ path/to/qnet15/converted/ --input_format=tf_saved_model --output_format=tfjs_graph_model
    

    This error can be silenced, however, by adding the --skip_op_check flag to the above command. It will generated the expected model.json with its corresponding weight binaries after a bunch of warnings.

    But, if you try inference @node, the same error occurs:

    Promise {
      <rejected> TypeError: Unknown op 'AudioSpectrogram'. File an issue at https://github.com/tensorflow/tfjs/issues so we can add it, or register a custom execution with tf.registerOp()
    

    The model is basically useless. There is an open issue for this feature since some years now.