pytorchcudagpuonnxruntime

ONNXRuntimeError: LoadLibrary failed with error 126 onnxruntime\capi\onnxruntime_providers_cuda.dll

CUDA does not seem to be used when I run my model.
Any help would be much appreciated!

2024-07-30 11:14:52.5255821 [E:onnxruntime:Default, provider_bridge_ort.cc:1745 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 ```
onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\david\miniconda3\envs\Pose2Sim\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

2024-07-30 11:14:52.5390456 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:895 onnxruntime::python::CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirementsto ensure all dependencies are met.
load C:\Users\david\.cache\rtmlib\hub\checkpoints\yolox_m_8xb8-300e_humanart-c2c7a14a.onnx with onnxruntime backend

I run Windows 11. I installed like so but got the same error when trying with pytorch with Cuda 12.4.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install onnxruntime-gpu

The following looks sensible and in accordance with the requirements here: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements.

import torch; import onnxruntime as ort; print(torch.cuda.is_available(),ort.get_available_providers())
print(f'torch version: {torch.__version__}, cuda version: {torch.version.cuda}, cudnn version: {torch.backends.cudnn.version()}, onnxruntime version: {ort.__version__}')

True ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
torch version: 2.4.0+cu118, cuda version: 11.8, cudnn version: 90100, onnxruntime version: 1.18.1

Nvidia-smi gives this. I doubt the missing power information has anything to do with the issue but it's probably worth noting it, I could be wrong.

enter image description here

Dependency walker on the missing library gives this. I suppose this is the root of my problem, but I'm not sure what to do with it. enter image description here

Solution

  • I got it in the end: I installed pytorch 2.4.0+cu118, which uses cuDNN 9. However, cuDNN 9 is not supported by ONNXruntime-gpu 1.18.1+cu11.8, so I needed to install pytorch 2.3.1 instead, which ships with cuDNN 8:

    pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118

    Useful links: