pythonjupyter-notebookcondaintel-modinintel-cloud

Intel Modin Jupyter notebook issues

I have got two issues with modin

  1. Lately I am not able to create modin environment using the command:

    conda create -n aikit-modin intel-aikit-modin -c intel -c conda-forge

(as mentioned in the documentation link of AI Analytics toolkit https://software.intel.com/content/www/us/en/develop/articles/installing-ai-kit-with-conda.html#gs.2irfv6 )

It seems documentation correction is required.

  1. I was able to create modin environment using stock

    conda create -n stock-modin modin -c conda-forge

(referred-https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/distribution-of-modin.html#gs.2isi2d and Data Science at Scale with Modin documentation:

https://medium.com/intel-analytics-software/data-science-at-scale-with-modin-5319175e6b9a )

But while trying to do the modin sample, NYC_Taxi.ipynb, in the online JupyterLab of Intel devcloud, I am getting a

file save server error JupyterHub server no longer running at /user/uXXXXX/api/contents/modin/examples/jupyter/NYC_Taxi.ipynb. Restart the server at http://jupyter.oneapi.devcloud.intel.com/hub/spawn/uXXXXX

as shown in the screenshot. I was not able to complete the notebook. It seems like a kernel error, even though I created a fresh kernel from the stock-modin environment.

Screenshot

(source of the NYC sample:https://github.com/modin-project/modin/tree/master/examples )

How to execute the samples of Intel Distribution of Modin? Any workarounds?

Solution
    1. conda create -n aikit-modin intel-aikit-modin -c intel -c conda-forge

    This command takes time to execute. As you can create modin environment using stock, you can execute the notebook using the below method.

    1. As a workaround, devcloud tunneling is a solution to this kernel error. Open terminals like Cygwin, vscode, mobaxterm or openssh. Setting up of devcloud connection should be done before going to the steps below. Please refer Connect to the Devcloud option in https://devcloud.intel.com/oneapi/documentation/shell-commands/

    If you have connected to devcloud using any of the above terminals, please follow the below steps.

    In the first terminal execute the following commands

    ssh devcloud  //(To connect to devcloud)
qsub -I  //(to request compute node)
copy the compute node  //(it would be of this format sXXX-nXXX)
conda create -n stock-modin modin -c conda-forge  //(to create modin environment using stock)
source activate stock-modin  //(activate modin environment)
conda install pip
conda install ipykernel  //(package for creating kernels)
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"  //(command to create kernels)
jupyter notebook --no-browser --port=<desired_port_number>  //(to launch Jupyter)
Copy any one of the URLs
“Or copy and paste one of these URLs:
http://localhost: <desired_port_number>/?token=2deea502bd258ed648f59eb11383bff4fd0edf5a4ce4f4f3or 
http://127.0.0.1: <desired_port_number>/?token=2deea502bd258ed648f59eb11383bff4fd0edf5a4ce4f4f3”

    Run the below commands in another terminal: (Keep the first terminal open)

    ssh -L <desired_port_number>:localhost: <desired_port_number> devcloud 
ssh -L <desired_port_number>:localhost: <desired_port_number> <compute node you copied>
    1. Open the copied URL in a browser
    2. You’ll get Jupyter, open the required notebook.
    3. Select the kernel that you created
    4. Execute the notebook.