kubeflowkubeflow-pipelinestfx

Equivalent of TFX Standard Components in KubeFlow

I have an existing TFX pipeline here that I want to rewrite using the KubeFlow Pipelines SDK.

The existing pipeline is using many TFX Standard Components such as ExampleValidator. When checking the KubeFlow SDK, I see a kfp.components.package but no existing prebuilt components like TFX provides.

Does the KubeFlow SDK have an equivalent to the TFX Standard Components?

Solution

  • You don’t have to rewrite the components, there is no mapping of components of tfx in kfp, as they are not competitive tools.

    With tfx you create the components and then you use an orchestrator to run them. Kubeflow pipelines is one of the orchestrators.

    The tfx.orchestration.pipeline will wrap your tfx components and create your pipeline.

    We have two schedulers behind kubeflow pipelines: Argo (used by gcp) and Tekton (used by openshift). There are examples for tfx with kubeflow pipelines using tekton and tfx with kubeflow pipelines using argo in the respective repositories.