pytorchpycharmcudajetbrains-ide

PyCharm 2025.1 Jupyter kernel fails on import torch with ImportError: libnccl.so.2, but same virtual-env works in terminal


I am running into the weirdest error.

Using PyCharm's Jupyter Notebook IDE (2025.1) I get the following error when I run import torch

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 import torch

File ~/.local/lib/python3.10/site-packages/torch/__init__.py:237
    235     if USE_GLOBAL_DEPS:
    236         _load_global_deps()
--> 237     from torch._C import *  # noqa: F403
    239 # Appease the type checker; ordinarily this binding is inserted by the
    240 # torch._C module initialization code in C
    241 if TYPE_CHECKING:

ImportError: libnccl.so.2: cannot open shared object file: No such file or directory

However, when I run ipython in the terminal using the same virtual environment I get no issues:

Python 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.11.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import torch

In [2]: torch.__version__
Out[2]: '2.2.2+cu121'

Any ideas of what may be causing that error?

EDIT 1: Search Paths

The search paths in the terminal and in PyCharm are the same: (using set(os.getenv("LD_LIBRARY_PATH").split(":")) to make them clearer)

PyCharm: {'/usr/local/cuda-12.2/lib64', '/usr/local/cuda-12.2/extras/CUPTI/lib64'} Terminal: {'/usr/local/cuda-12.2/lib64', '/usr/local/cuda-12.2/extras/CUPTI/lib64'}

Thus, that cannot be the difference between the two.


Solution

  • After some digging I found the issue.

    TLDR: if you are using conda conda install -c nvidia nccl solves it. (my case).

    If you interested on what went wrong:
    The libnccl.so.2 shared library is inside PyTorch’s site-packages tree, not in any default search path.

    ImportError: libnccl.so.2: cannot open shared object file: No such file or directory
    

    The issue is that the expected file is somewhere else!

    # where the library really is
    $ find /usr /usr/local -name 'libnccl.so.2'
    /usr/local/lib/python3.10/dist-packages/nvidia/nccl/lib/libnccl.so.2
    

    Fixes

    Any of the three options below unblocks import torch in the PyCharm Jupyter kernel.

    1 Add the directory to the system linker path (recommended if you have sudo)

    echo /usr/local/lib/python3.10/dist-packages/nvidia/nccl/lib \
      | sudo tee /etc/ld.so.conf.d/nccl.conf
    sudo ldconfig            # refresh the linker cache
    

    2 Export LD_LIBRARY_PATH and let PyCharm see it

    # ~/.bashrc  (or ~/.zshrc)
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/python3.10/dist-packages/nvidia/nccl/lib
    

    PyCharm » Settings ▸ Build, Execution & Deployment ▸ Target Environments
    → your SSH target → Environment variables → add the same LD_LIBRARY_PATH.

    3 Launch the kernel from the env root

    PyCharm » Settings ▸ Jupyter → edit your remote interpreter


    After applying one of those solutions restart the remote kernel; import torch should succeed. It worked for me!