I try to run NVIDIA’s Triton Inference Server. I pulled the pre-built container nvcr.io/nvidia/pytorch:22.06-py3 and then run it with the command
run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/F/models:/models nvcr.io/nvidia/pytorch:22.06-py3 tritonserver --model-repository=/models
and got the error
/opt/nvidia/nvidia_entrypoint.sh: line 49: exec: tritonserver: not found
I googled and have not found something to catch this. I tried to change tritonserver to trtserver as recommended but it did not help. Please give some advice how it can be solved.
Looks like you're trying to run a tritonserver using a pytorch image but according to the triton-server quick start guide, the image should be:
$ docker run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/full/path/to/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:<xx.yy>-py3 tritonserver --model-repository=/models
Where <xx.yy> is the version of Triton that you want to use
In your case it should be nvcr.io/nvidia/tritonserver:22.06-py3
and the full command:
run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/F/models:/models nvcr.io/nvidia/tritonserver:22.06-py3 tritonserver --model-repository=/models