kuberneteskubebuilder

How to include the spec of a CRD into another CRD?


I am building an operator using Kubebuilder which will work with a CRD which contains a few custom fields and also all fields of an Argo WorkflowTemplate. The fields will be essentially all the fields of an Argo Workflow as the fields in the spec are the same apart from one field which I don't care about. The operator will use the WorkflowTemplate fields to create an actual Argo WorkflowTemplate resource, but before that it needs to perform some actions on the custom fields. How can I include all the fields of the WorkflowTemplate in my CRD without literally copying all the fields and manually pasting it in the Spec of my CRD?

I tried having this in my CRD(I just added certain other fields that are not related to the problem at hand)

import (
    wfv1 "github.com/argoproj/argo-workflows/v3/pkg/apis/workflow/v1alpha1"
)

type DefinitionSpec struct {
    Field1       string            `json:"field1,omitempty"`
    Field2       string            `json:"field2,omitempty"`
    WorkflowSpec wfv1.WorkflowSpec `json:",inline"`
}

If I do this, I get this error while installing the CRD

$ make install
/Users/mbtamuli/workspace/argo-workflow-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/mbtamuli/workspace/argo-workflow-operator/bin/kustomize build config/crd | kubectl apply -f -
The CustomResourceDefinition "definitions.mriyam.dev" is invalid:
* metadata.annotations: Too long: must have at most 262144 bytes
* spec.validation.openAPIV3Schema.properties[spec].properties[templateDefaults].properties[dag].properties[tasks].items.properties[inline].type: Required value: must not be empty for specified object fields
* spec.validation.openAPIV3Schema.properties[spec].properties[templateDefaults].properties[steps].items.items: Required value: must be specified
* spec.validation.openAPIV3Schema.properties[spec].properties[templates].items.properties[dag].properties[tasks].items.properties[inline].type: Required value: must not be empty for specified object fields
* spec.validation.openAPIV3Schema.properties[spec].properties[templates].items.properties[steps].items.items: Required value: must be specified
make: *** [install] Error 1

I found out from other SO and GitHub issues that I can get around the annotations issue by using create instead of apply.

$ bin/kustomize build config/crd | kubectl create -f -
+ kubectl create -f -
The CustomResourceDefinition "definitions.mriyam.dev" is invalid:
* spec.validation.openAPIV3Schema.properties[spec].properties[templateDefaults].properties[dag].properties[tasks].items.properties[inline].type: Required value: must not be empty for specified object fields
* spec.validation.openAPIV3Schema.properties[spec].properties[templateDefaults].properties[steps].items.items: Required value: must be specified
* spec.validation.openAPIV3Schema.properties[spec].properties[templates].items.properties[dag].properties[tasks].items.properties[inline].type: Required value: must not be empty for specified object fields
* spec.validation.openAPIV3Schema.properties[spec].properties[templates].items.properties[steps].items.items: Required value: must be specified

Solution

  • I had exactly the same problem when wrapping the Argo Workflows CRD and then I discovered that the Argo Workflows project avoids the issues by removing CRD validation:

    https://github.com/argoproj/argo-workflows/blob/master/hack/crds.go

    The most important part is the following function:

    func removeCRDValidation(filename string) {
        data, err := ioutil.ReadFile(filepath.Clean(filename))
        if err != nil {
            panic(err)
        }
        crd := make(obj)
        err = yaml.Unmarshal(data, &crd)
        if err != nil {
            panic(err)
        }
        spec := crd["spec"].(obj)
        versions := spec["versions"].([]interface{})
        version := versions[0].(obj)
        properties := version["schema"].(obj)["openAPIV3Schema"].(obj)["properties"].(obj)
        for k := range properties {
            if k == "spec" || k == "status" {
                properties[k] = obj{"type": "object", "x-kubernetes-preserve-unknown-fields": true, "x-kubernetes-map-type": "atomic"}
            }
        }
        data, err = yaml.Marshal(crd)
        if err != nil {
            panic(err)
        }
        err = ioutil.WriteFile(filename, data, 0o600)
        if err != nil {
            panic(err)
        }
    }
    

    Essentially they override the CRD spec, and set x-kubernetes-preserve-unknown-fields to true to ensure that the Kubernetes API accepts requests without checking the fields.