pytorchhuggingfaceonnxonnxruntimeopenai-whisper

HuggingFace Model - OnnxRuntime - Jupyter Notebook Print Model Summary


Thank you very much for reading my question , sorry if it is an obvious question.

I use anaconda navigator : piped install the model whisper from OpenAi, which is an audio to text transformer model, I use jupyter notebook and when I just run the cell of the model, there is this summary of modules which is quite useful to get to know what the model is :


enter image description here


However, with another pip installed model : https://huggingface.co/breezedeus/pix2text-mfr I notice the difference is it is optimum.onnxruntime

enter image description here


and when I do the same thing as above, it instead returns a memory location ? or is it ?

sorry if it is a simple question, I tried googling a bit but dont quite know what keyword to search - "onnx pytorch model summary" ? is there a way to have a model summary as above ?

Thank you very much for reading my question .


Solution

  • The ORT model output is just the default string that represents an object in Python, providing the class name and then the memory address. They are both valid objects in Python but the first model overrides the __str__() method to show the layers when the model is printed.

    The ORT model doesn't have the same methods, like the __str__() override, as the whisper model because it is not a PyTorch-based model. Instead, it uses ONNX graph definitions and operators. You can use a tool like Netron. You can try to print the encoder and decoder graphs, but it is not very readable.

    import onnx
    
    # Encoder model
    encoder_onnx_model = onnx.load(model.encoder_model_path)
    print(onnx.helper.printable_graph(encoder_onnx_model.graph))
    
    # Decoder model
    decoder_onnx_model = onnx.load(model.decoder_model_path)
    print(onnx.helper.printable_graph(decoder_onnx_model.graph))