I have created this Conda environment:
conda env create -f environment.yml
The environment.yml
file:
name: deep3d_pytorch
channels:
- pytorch
- conda-forge
- defaults
dependencies:
- python=3.6
- pytorch=1.6.0
- torchvision=0.7.0
- numpy=1.18.1
- scikit-image=0.16.2
- scipy=1.4.1
- pillow=6.2.1
- pip
- ipython=7.13.0
- yaml=0.1.7
- pip:
- matplotlib==2.2.5
- opencv-python==3.4.9.33
- tensorboard==1.15.0
- tensorflow==1.15.0
- kornia==0.5.5
- dominate==2.6.0
- trimesh==3.9.20
I activate the Conda environment. But even a simple statement like python -c "import torch; print(torch.__version__)"
to get the PyTorch version throws the undefined symbol
error:
(deep3d_pytorch) m3@i7:~/repos/Deep3DFaceRecon_pytorch> python -c "import torch; print(torch.__version__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/m3/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/__init__.py", line 189, in <module>
from torch._C import *
ImportError: /home/m3/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_IsProfilingActive
I believe the PyTorch installed by Conda is broken. But the Conda logs are all fine. Does anyone have a clue or hint? I'm receiving the undefined symbol
error on both my local machine and on Google Colab.
Even a minimal Environment like below, would throw similar errors:
conda create -n minimal_pytorch python=3.6 pytorch torchvision torchaudio -c pytorch
source activate minimal_pytorch && python -c "import torch; print(torch.__version__)"
A similar undefined symbol
error is thrown:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/envs/minimal_pytorch/lib/python3.6/site-packages/torch/__init__.py", line 197, in <module>
from torch._C import * # noqa: F403
ImportError: /usr/local/envs/minimal_pytorch/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent
When Python version is omitted while creating the environment:
conda create -n minimal_pytorch python pytorch torchvision torchaudio -c pytorch
The error is resolved:
source activate minimal_pytorch && python -c "import torch; print(torch.__version__)"
PyTorch version is received without any error:
2.2.2
To show how it was solved, I post all our code here. We were trying to test this repository:
https://github.com/sicxu/Deep3DFaceRecon_pytorch
Our final approach on Google Colab that works just fine is:
# Step 1: Press runtime on the top > Change runtime type > Select T4 GPU
# Step 2: Run this shell
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
!bash miniconda.sh -b -u -p /usr/local
!rm miniconda.sh
!conda update -y conda
%cd /content/
!rm -rf Deep3DFaceRecon_pytorch
!git clone https://github.com/sicxu/Deep3DFaceRecon_pytorch.git
%cd Deep3DFaceRecon_pytorch
!git clone https://github.com/NVlabs/nvdiffrast
!git clone https://github.com/deepinsight/insightface.git
!cp -r ./insightface/recognition/arcface_torch ./models/
# Step 3: Mount drive and copy the files mounted in google drive:
!mkdir /content/Deep3DFaceRecon_pytorch/checkpoints
!cp -r /content/drive/MyDrive/Deep3D/facerecon_20230425 /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425
!cp /content/drive/MyDrive/Deep3D/01_MorphableModel.mat /content/Deep3DFaceRecon_pytorch/BFM/
!cp /content/drive/MyDrive/Deep3D/Exp_Pca.bin /content/Deep3DFaceRecon_pytorch/BFM/
# Step 4: Install the graphics related stuff
!sudo apt-get install libegl1-mesa-dev
!nvidia-smi
!sudo apt-get install libnvidia-gl-535
# Step 5: Run this shell to install everything
%%shell
eval "$(conda shell.bash hook)"
conda create --name deep3d_pytorch python=3.6 -y
conda activate deep3d_pytorch
conda config --env --add channels pytorch
conda config --env --add channels conda-forge
conda config --env --add channels defaults
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch -y
conda install numpy scikit-image=0.16.2 scipy=1.4.1 pillow=6.2.1 pip ipython=7.13.0 yaml=0.1.7 -y
pip install matplotlib==2.2.5 opencv-python==3.4.9.33 tensorboard==1.15.0 tensorflow==1.15.0 kornia==0.5.5 dominate==2.6.0 trimesh==3.9.20
pip install ./nvdiffrast/.
# Step 6: Run this shell to test the program
%%shell
eval "$(conda shell.bash hook)"
conda activate deep3d_pytorch
python test.py --name=facerecon_20230425 --epoch=20 --img_folder=./datasets/examples
# Step 7: Copy test results to Google Drive
!ls /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/examples/epoch_20_000000/
!cp /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/examples/epoch_20_000000/* /content/drive/MyDrive/Deep3D/results/
# Step 8: prepare custom images along with their facial landmarks
!cp -r /content/drive/MyDrive/Deep3D/custom_images /content/Deep3DFaceRecon_pytorch/
%cd /content/Deep3DFaceRecon_pytorch/custom_images/
!python -m venv virtual_env
!source virtual_env/bin/activate && which pip
!source virtual_env/bin/activate && pip install mtcnn
!source virtual_env/bin/activate && pip install tensorflow
!source virtual_env/bin/activate && which python
!source virtual_env/bin/activate && python facial_landmarks.py # This Python script source code can be found here: https://github.com/sicxu/Deep3DFaceRecon_pytorch/issues/85#issuecomment-2069302718
%cd /content/Deep3DFaceRecon_pytorch/
# Step 9: run with custom images
%%shell
eval "$(conda shell.bash hook)"
conda activate deep3d_pytorch
python test.py --name=facerecon_20230425 --epoch=20 --img_folder=./custom_images
# Step 10: copy the results back to Google Drive
!ls /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/custom_images/epoch_20_000000/
!cp /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/custom_images/epoch_20_000000/* /content/drive/MyDrive/Deep3D/results/
It's described here too: https://github.com/conda/conda/issues/13812#issuecomment-2071445372