[SOLVED] Connecting to AI Platform Pipelines using the Kubeflow Pipelines SDK with Python and PyCharm

Connecting to AI Platform Pipelines using the Kubeflow Pipelines SDK with Python and PyCharm

Using windows 10 and Python 3.9 with PyCharm IDE

I am trying to list the pipelines using the following code:

import kfp
client = kfp.Client(host='.......bb92ea05-dot-us-central2.pipelines.googleusercontent.com')
    
client.list_pipelines()

I know the host variable is correct because I copied it from the AI platform Pipeline. I am getting the following problem:

FileNotFoundError: [WinError 2] The system cannot find the file specified

I think that the file can't be found since the program can't connect to the GCP's AI PLatform Pipeline.

I have installed Kubeflow Pipeline SDK by doing the following commands in Pycharm terminal:

conda create --name mlpipeline python=3.7
conda activate mlpipeline
pip3 install kfp --upgrade --user

When I run:

kfp diagnose_me

I get the following command error:

Google Cloud SDK is not installed, gcloud, gsutil and kubectl are required for this app to run. Please follow instructions at https://cloud.google.com/sdk/install to install the SDK

I have tried

Installing Google Cloud SDK and initializing gcloud(Although when I run a command it says 'Python not found')
Installing kubectl and gsutil with gcloud commands

Solution

I transitioned over to Ubuntu and from there the problem was fixed.

What I did:

Installed Google Cloud SDK and configured it: https://cloud.google.com/sdk/docs/quickstart

Also install gsutil and kubectl using gcloud components install component_id command

Install KFP SDK: Run the following commands in terminal:

 apt-get update; apt-get install -y wget bzip2
 wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
 bash Miniconda3-latest-Linux-x86_64.sh

 conda create --name mlpipeline python=3.7   
 conda activate mlpipeline

 pip3 install kfp --upgrade