[SOLVED] Torch 2 installed - Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory

Torch 2 installed - Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory

I have installed Torch 2 via this command on RunPod io instance

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Everything installed perfectly fine

With Torch 1 and Cuda 11.7, I was not getting any error but with Torch 2 the below error produced

Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory

How to fix?

It is using unix

On Windows same prodecure working very well

Using Automatic1111 web UI to use Stable Diffusion

Solution

This appears to be an issue with the cuda libraries packaged with Torch 2. Specifically, the libnvrtc.so library is missing from the latest Torch distribution.

To fix this, I did the following:

Download the relevant CUDA libraries. I used CUDA 11.8, by running: "wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run"
Install the CUDA libraries by running the installer. Deselect the option to install the drivers when presented, as this is not needed. In my case the CUDA libraries are now installed in /usr/local/cuda/lib64
Add the path to the CUDA library to you LD_LIBRARY_PATH, by adding "export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH" to your .bashrc file.

This solved the problem for me, but does mean that my system is now bypassing the CUDA libraries that were installed with Pytorch 2 and only using the CUDA libraries I installed.

Presumably this issue will be fixed with subsequent releases of the compiled Pytorch 2 library.