pytorchtensorrt

when I use the trtexec --onnx=** --saveEngine=** to transfer my onnx to a trt model, I got a warning from TensorRT

My workflow is like: pytorch --> onnx --> trt. I use torch.onnx.export() function to export my model with a FP16 precision. And then I use the trtexec --onnx=** --saveEngine=** to transfer my onnx file to a trt model,a warning came out like: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. I don't know why and does anyone who can help me with this?

I tried what I can to solve this, but seemly noting can work.

Solution

  • Inference with explicit batch onnx model, try:

    trtexec --explicitBatch --onnx=your_model.onnx

    https://elinux.org/TensorRT/ONNX#How_to_use_trtexec_to_run_inference_with_dynamic_shape.3F