kuberneteskustomizeopen-policy-agent

Is there a way to have kustomize merge the content of a file into a yaml file?


I am curious to know if there is a way to do a Kustomize replacement or other operation to inject the contents of a non-yaml file into a yaml file using Kustomize. I know Kustomize is not a template engine and that this could be accomplished with Helm, but using the tool I am already using, is this possible?

My use case is to store OPA policies as native rego, which allows use of OPA unit tests, and to inject the content of these rego files into Gatekeeper constraints during Kustomize deployment. This will remove the requirement for custom pipeline processing or manual copy/paste to accomplish this.

Example opaRule.rego file

package k8sdisallowedtags

violation[{"msg": msg}] {
    container := input_containers[_]
    tags := [forbid | tag = input.parameters.tags[_] ; forbid = endswith(container.image, concat(":", ["", tag]))]
    any(tags)
    msg := sprintf("container <%v> uses a disallowed tag <%v>; disallowed tags are %v", [container.name, container.image, input.parameters.tags])
}
...

Example constraintTemplate.yaml file

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sdisallowedtags
  namespace: kube-system
  annotations:
    description: Requires container images to have an image tag different
      from the ones in a specified list.
spec:
  crd:
    spec:
      names:
        kind: K8sDisallowedTags
      validation:
        openAPIV3Schema:
          properties:
            tags:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |-
        {CONTENT OF OPA RULE POLICY HERE}

Solution

  • Answer will contain two parts:

    Create own plugin

    Kustomize allows to create plugins to extend its functionality. And there are almost zero limitations including security - this should be handled by author of the plugin.

    There are two kinds of plugins:

    All available information can be found in Extending Kustomize - documentation.

    Example of exec plugin. Note! correct flag is --enable-alpha-plugins (with -, not with _ as in example).

    Using patches

    Patches (also call overlays) add or override fields on resources. They are provided using the patches Kustomization field.

    The patches field contains a list of patches to be applied in the order they are specified.

    Each patch may:

    • be either a strategic merge patch, or a JSON6902 patch
    • be either a file, or an inline string target
    • a single resource or multiple resources

    Reference to kustomize - patches

    Below is example how to patch gatekeeper.yaml object.

    Structure:

    $ tree

    .
    ├── gatekeeper.yaml
    ├── kustomization.yaml
    └── opa-gk.yaml
    

    $ cat gatekeeper.yaml

    apiVersion: templates.gatekeeper.sh/v1beta1
    kind: ConstraintTemplate
    metadata:
      name: k8sdisallowedtags
      namespace: kube-system
      annotations:
        description: Requires container images to have an image tag different
          from the ones in a specified list.
    spec:
      crd:
        spec:
          names:
            kind: K8sDisallowedTags
          validation:
            openAPIV3Schema:
              properties:
                tags:
                  type: array
                  items:
                    type: string
      targets:
        - target: admission.k8s.gatekeeper.sh
          rego: |-
            {CONTENT OF OPA RULE POLICY HERE}
    

    $ cat kustomization.yaml

    resources:
    - gatekeeper.yaml
    
    patches:
    - path: opa-gk.yaml
      target:
        group: templates.gatekeeper.sh
        version: v1beta1
        kind: ConstraintTemplate
        name: k8sdisallowedtags
    

    $ cat opa-gk.yaml

    - op: add
      path: /spec/targets/0/rego
      value: |
        package k8sdisallowedtags
        
        violation[{"msg": msg}] {
            container := input_containers[_]
            tags := [forbid | tag = input.parameters.tags[_] ; forbid = endswith(container.image, concat(":", ["", tag]))]
            any(tags)
            msg := sprintf("container <%v> uses a disallowed tag <%v>; disallowed tags are %v", [container.name, container.image, input.parameters.tags])
        }
        ...
    

    End result:

    $ kubectl kustomize .

    apiVersion: templates.gatekeeper.sh/v1beta1
    kind: ConstraintTemplate
    metadata:
      annotations:
        description: Requires container images to have an image tag different from the
          ones in a specified list.
      name: k8sdisallowedtags
      namespace: kube-system
    spec:
      crd:
        spec:
          names:
            kind: K8sDisallowedTags
          validation:
            openAPIV3Schema:
              properties:
                tags:
                  items:
                    type: string
                  type: array
      targets:
      - rego: |
          package k8sdisallowedtags
    
          violation[{"msg": msg}] {
              container := input_containers[_]
              tags := [forbid | tag = input.parameters.tags[_] ; forbid = endswith(container.image, concat(":", ["", tag]))]
              any(tags)
              msg := sprintf("container <%v> uses a disallowed tag <%v>; disallowed tags are %v", [container.name, container.image, input.parameters.tags])
          }
          ...
        target: admission.k8s.gatekeeper.sh