I connect to destination machine (ada) like this:
connect to globalprotect vpn
ssh -X machine 1
(machine 1) : ssh -X ada
my own machine is Ubuntu and the other two I ssh to also same version of Ubuntu. How can I fix this error?
(gdrnpp) mona@ada:~/gdrnpp_bop2022$ ./core/gdrn_modeling/train_gdrn.sh configs/gdrn/ycbv/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_ycbv.py 0
++ dirname ./core/gdrn_modeling/train_gdrn.sh
+ this_dir=./core/gdrn_modeling
+ CFG=configs/gdrn/ycbv/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_ycbv.py
+ CUDA_VISIBLE_DEVICES=0
+ IFS=,
+ read -ra GPUS
+ NGPU=1
+ echo 'use gpu ids: 0 num gpus: 1'
use gpu ids: 0 num gpus: 1
+ NCCL_DEBUG=INFO
+ OMP_NUM_THREADS=1
+ MKL_NUM_THREADS=1
+ PYTHONPATH=./core/gdrn_modeling/../..:/home/mona/realsense-ros/install/realsense2_camera_msgs/local/lib/python3.10/dist-packages:/opt/ros/humble/lib/python3.10/site-packages:/opt/ros/humble/local/lib/python3.10/dist-packages
+ CUDA_VISIBLE_DEVICES=0
+ python ./core/gdrn_modeling/main_gdrn.py --config-file configs/gdrn/ycbv/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_ycbv.py --num-gpus 1
/home/mona/.local/lib/python3.10/site-packages/mmcv/__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
warnings.warn(
You requested to import horovod which is missing or not supported for your OS.
/home/mona/.local/lib/python3.10/site-packages/mmcv/device/npu/data_parallel.py:22: UserWarning: Torchaudio's I/O functions now support par-call bakcend dispatch. Importing backend implementation directly is no longer guaranteed to work. Please use `backend` keyword with load/save/info function, instead of calling the udnerlying implementation directly.
if hasattr(sys.modules[m], '_check_balance'):
/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/pysixd/misc.py:586: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def get_obj_im_c(K, t):
/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/pysixd/misc.py:765: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def compute_2d_bbox_xyxy_from_pose(points, pose, K, width=640, height=480, clip=False):
/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/pysixd/misc.py:793: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def compute_2d_bbox_xyxy_from_pose_v2(points, pose, K, width=640, height=480, clip=False):
/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/pysixd/misc.py:822: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def compute_2d_bbox_xywh_from_pose(points, pose, K, width=640, height=480, clip=False):
Traceback (most recent call last):
File "/home/mona/gdrnpp_bop2022/./core/gdrn_modeling/main_gdrn.py", line 39, in <module>
from core.gdrn_modeling.engine.engine_utils import get_renderer
File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/engine_utils.py", line 7, in <module>
from lib.egl_renderer.egl_renderer_v3 import EGLRenderer
File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/egl_renderer/egl_renderer_v3.py", line 27, in <module>
from . import CppEGLRenderer
ImportError: cannot import name 'CppEGLRenderer' from 'lib.egl_renderer' (/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../lib/egl_renderer/__init__.py)
I have:
(base) mona@ada:~$ uname -a
Linux ada 6.2.0-37-generic #38~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 2 18:01:13 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(base) mona@ada:~$ lsb_release -a
LSB Version: core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
(base) mona@ada:~$ nvidia-smi
Fri Dec 22 14:24:54 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.12 Driver Version: 535.104.12 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX 6000 Ada Gene... On | 00000000:52:00.0 Off | Off |
| 31% 60C P2 71W / 300W | 4213MiB / 49140MiB | 1% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 2310 G /usr/lib/xorg/Xorg 913MiB |
| 0 N/A N/A 2535 G /usr/bin/gnome-shell 80MiB |
| 0 N/A N/A 3040 G ...AAAAAAAACAAAAAAAAAA= --shared-files 43MiB |
| 0 N/A N/A 26577 G /usr/lib/thunderbird/thunderbird 19MiB |
| 0 N/A N/A 151053 G ...AAAAAAAIAAAAAAAAAA== --shared-files 102MiB |
| 0 N/A N/A 862586 C python 918MiB |
| 0 N/A N/A 863180 C python 730MiB |
| 0 N/A N/A 864122 C python 730MiB |
| 0 N/A N/A 901572 C+G ./blender 411MiB |
| 0 N/A N/A 1014271 G ...sion,SpareRendererForSitePerProcess 51MiB |
| 0 N/A N/A 1023738 G meshlab 13MiB |
| 0 N/A N/A 3651908 G ...irefox/3358/usr/lib/firefox/firefox 68MiB |
+-------------------------------------------------------------------------------
(gdrnpp) mona@ada:~/gdrnpp_bop2022$ cd lib/egl_renderer/
(gdrnpp) mona@ada:~/gdrnpp_bop2022/lib/egl_renderer$ pip install -e .