I want to create kubeflow pipelines file to run pipeline on GKE.
I created the pipeline according to
Getting started building a pipeline.
I cannot get kubeflow pipeline yaml file by following method.
kfp.compiler.Compiler().compile(
pipeline_func=my_pipeline,
package_path='pipeline.yaml',
pipeline_parameters={'url': 'https://storage.googleapis.com/ml-pipeline-playground/iris-csv-files.tar.gz'})
I got TypeError: compile() got an unexpected keyword argument 'pipeline_parameters'
From this implementation, I should be able to use pipeline_parameters
argument for kfp.compiler.Compiler().compile
.
If you want to pass default pipeline parameters for your pipeline. You can do it directly in the function definition of the pipeline like this:
@dsl.pipeline(name='name')
def my_pipeline(a=1, b='default value'):
# pipeline code
pass
For a minimal example of a full pipeline, you can take a look at this. From my experience working with Kubeflow, a lot of the documentation are wrong or not up-to-date. Many times, you have to find a workaround.