Getting ModuleNotFoundError for both 'azureml.pipeline' and azureml.widgets

While trying to import PythonScriptStep and RunDetails using:

from azureml.pipeline.steps import PythonScriptStep
from azureml.widgets import RunDetails

Getting ModuleNotFoundError: No module named 'azureml.pipeline' and ModuleNotFoundError: No module named 'azureml.widgets' errors

I tried manually installing azureml pipeline using:

!pip install azureml-pipeline

Getting various incompatability errors:

ERROR: ray 2.0.0 has requirement click<=8.0.4,>=7.0, but you'll have click 8.1.3 which is incompatible.
ERROR: ray 2.0.0 has requirement grpcio<=1.43.0,>=1.28.1; python_version < "3.10", but you'll have grpcio 1.54.2 which is incompatible.
ERROR: pyopenssl 23.0.0 has requirement cryptography<40,>=38.0.0, but you'll have cryptography 42.0.8 which is incompatible.
ERROR: jupyterlab-server 2.23.0 has requirement jinja2>=3.0.3, but you'll have jinja2 2.11.2 which is incompatible.
ERROR: datasets 2.3.2 has requirement dill<0.3.6, but you'll have dill 0.3.6 which is incompatible.
ERROR: dask-sql 2023.6.0 has requirement pandas>=1.4.0, but you'll have pandas 1.1.5 which is incompatible.
ERROR: azureml-widgets 1.56.0 has requirement azureml-core~=1.56.0, but you'll have azureml-core 1.51.0.post1 which is incompatible.
ERROR: azureml-widgets 1.56.0 has requirement azureml-telemetry~=1.56.0, but you'll have azureml-telemetry 1.51.0 which is incompatible.
ERROR: azureml-inference-server-http 0.8.4 has requirement flask<2.3.0, but you'll have flask 2.3.2 which is incompatible.
ERROR: azure-cli 2.49.0 has requirement azure-keyvault-keys==4.8.0b2, but you'll have azure-keyvault-keys 4.8.0 which is incompatible.
ERROR: azure-cli 2.49.0 has requirement azure-mgmt-keyvault==10.2.0, but you'll have azure-mgmt-keyvault 10.2.1 which is incompatible.
ERROR: azure-cli 2.49.0 has requirement azure-mgmt-resource==22.0.0, but you'll have azure-mgmt-resource 21.1.0b1 which is incompatible.
ERROR: azure-cli-core 2.49.0 has requirement msal[broker]==1.20.0, but you'll have msal 1.22.0 which is incompatible.
ERROR: autokeras 1.0.16 has requirement tensorflow<=2.5.0,>=2.3.0, but you'll have tensorflow 2.11.0 which is…
Show all (20.4 kB)

Can someone please help me with these errors.

If you are working in ml workspace you change the kernel to azureml_py38 which is sdk v1 as shown in below image.

If you want to create the separate environment locally, then create an yml file out this environment using below command in terminal.

conda env export > environment.yml

Next, create separate conda environment with that yml file using below command.

conda env create -f environment.yml