pandaslinuxanacondalarge-datamodin

Installing Modin Pandas in Linux (CentOS)

I am trying to install Modin on a shared computer which runs linux where I can get access to the terminal and jupyter. I created a virtual environment in Conda and used the steps mentioned in the official documentation for Modin. However when I tried to import Modin in the JupyterLab environment it is giving me these errors

#Executed Codeblock
import time
import modin.pandas as pd

import ray
ray.init(runtime_env={'env_vars': {'__MODIN_AUTOIMPORT_PANDAS__': '1'}})

Error :


2023-07-16 20:42:07,599 ERROR services.py:1169 -- Failed to start the dashboard , return code 1
2023-07-16 20:42:07,602 ERROR services.py:1194 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2023-07-16 20:42:07,604 ERROR services.py:1238 -- 
The last 20 lines of /tmp/ray/session_2023-07-16_20-42-04_202116_30517/logs/dashboard.log (it contains the error message from the dashboard): 
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/ray/dashboard/head.py", line 204, in _load_modules
    head_cls_list = dashboard_utils.get_all_modules(DashboardHeadModule)
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/ray/dashboard/utils.py", line 121, in get_all_modules
    importlib.import_module(name)
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/ray/dashboard/modules/job/cli.py", line 14, in <module>
    from ray.job_submission import JobStatus, JobSubmissionClient
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/ray/job_submission/__init__.py", line 2, in <module>
    from ray.dashboard.modules.job.pydantic_models import DriverInfo, JobDetails, JobType
  File "/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/ray/dashboard/modules/job/pydantic_models.py", line 4, in <module>
    from pydantic import BaseModel, Field
ImportError: cannot import name 'Field' from 'pydantic' (/home/verma.shi/.conda/envs/large/lib/python3.9/site-packages/pydantic/__init__.py)
2023-07-16 20:42:07,753 INFO worker.py:1553 -- Started a local Ray instance.
Ray
Python version: 3.9.16
Ray version:    2.3.0
(raylet) [2023-07-16 20:42:09,476 E 31350 31408] (raylet) agent_manager.cc:135: The raylet exited immediately because the Ray agent failed. The raylet fate shares with the agent. This can happen because the Ray agent was unexpectedly killed or failed. Agent can fail when
(raylet) - The version of `grpcio` doesn't follow Ray's requirement. Agent can segfault with the incorrect `grpcio` version. Check the grpcio version `pip freeze | grep grpcio`.
(raylet) - The agent failed to start because of unexpected error or port conflict. Read the log `cat /tmp/ray/session_latest/dashboard_agent.log`. You can find the log file structure here https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure.
(raylet) - The agent is killed by the OS (e.g., out of memory).

Can someone tell me what the exact issue for this is?

Solution

  • Versions of modin prior to 0.23.0 don't work with the latest pydantic. Try either: