kuberneteskubernetes-secretsconfiguration-managementkustomize

Use Kustomize Replacements to Replace Values in One Base with Values from Another Base?


I'm updating some of my Kubernetes configurations to use 'replacements' and 'resources' in kustomize as 'vars' and 'bases' have been deprecated.

Previously, I used 'vars' in a base (/base/secrets/) like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

secretGenerator:
- name: test_secret
  env: secret.env

vars:
- name : SECRET_VALUE
  objref:
    kind: Secret
    name: test_secret
    apiVersion: v1
  fieldref:
    fieldpath: metadata.name

This base was used in various overlays for different services:

namespace: test-overlay

bases:
- ../../base/secrets/
- ../../base/service/

Now, with 'resources' and 'replacements', my understanding is that it's not possible to replace values in /base/service/ from /base/secrets/ as before. I could apply the 'replacement' in the overlay itself and target the base I want to modify, but I would prefer to perform the operation from a base for reusability and ease of use.

Here's what I'm trying to do:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

secretGenerator:
- name: test_secret
  env: secret.env

replacements:
- source:
    name: test_secret
    kind: Secret
  targets:
  - select:
      kind: Deployment
      name: service
    fieldPaths:
    - spec.template.spec.<field>

In the 'replacements' directive, spec.template.spec.<field> is the field in the Deployment resource that I'm trying to replace.

I'm using kustomize version v5.1.0.

How can I get 'replacements' to target other bases so that they can be used from any overlay? What's the best practice for this scenario?

I've attempted to apply the 'replacements' in the overlay itself and target the base I want to modify like this:

namespace: test-overlay

resources:
- ../../base/secrets/
- ../../base/service/

replacements:
- source:
    kind: Secret
    name: test_secret
  targets:
  - select:
      kind: Deployment
      name: service
    fieldPaths:
    - spec.template.spec.<field>

While this does apply the replacement to the service, it's not a satisfactory solution for me because I have multiple overlays that all need to use the same replacement for various deployments. I would prefer to define the replacement once in a base, rather than having to define it in each overlay.

Edit: A more clear minimal reproducible example

/base
  /secrets
    kustomization.yaml
  /service
    deployment.yaml
    kustomization.yaml
/overlays
  /test-overlay
    kustomization.yaml

With the /secrets/ implemented as:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

secretGenerator:
- name: test_secret
  env: secret.env

replacements:
- source:
    name: test_secret
    kind: Secret
  targets:
  - select:
      kind: Deployment
      name: service
    fieldPaths:
    - spec.template.spec.volumes.name

This would be the /service/:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
    
resources:
- deployment.yaml

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: service
spec:
  template:
    spec:
      volumes:
      - name: placeholder_value
        emptyDir: {}

With /test-overlay/

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: test-overlay

resources:
- ../../base/secrets/
- ../../base/service/

I've tried including the 'replacements' in the test-overlays kustomization file which did work but is less desirable then letting the replacement be defined in a base


Solution

  • Thanks for updating your question with an example.

    The answer I've posted here is still the correct solution in terms of sharing a replacement configuration among multiple overlays, but there are some errors in your replacement syntax: you cannot target spec.template.spec.volumes.name, because volumes is a list and has no name attribute.

    You can only target list elements with a [name=value] style selector, so:

    replacements:
      - source:
          name: test_secret
          kind: Secret
        targets:
          - select:
              kind: Deployment
              name: service
            fieldPaths:
              - spec.template.spec.volumes.[name=placeholder_value].name
    

    A kustomization.yaml can only apply transformations (labels, patches, replacements, etc) to resources that are emitted by that kustomization.yaml -- which means that if you want a transformation to affect all resources, it needs to be applied in the "outermost" kustomization.

    This means that you can't place something in a "base" that will modify resources generated in your overlays.

    But don't worry, there is a solution! Components allow you to reuse kustomization fragments. If we move your replacement configuration into a component, we can get the behavior you want.

    For example, here is a project with a base and two overlays:

    .
    ├── base
    │   ├── deployment.yaml
    │   └── kustomization.yaml
    ├── components
    │   └── replace-username-password
    │       └── kustomization.yaml
    └── overlay
        ├── env1
        │   └── kustomization.yaml
        └── env2
            └── kustomization.yaml
    

    base/deployment.yaml looks like this:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: example
    spec:
      replicas: 2
      template:
        spec:
          containers:
            - name: example
              image: docker.io/alpine:latest
              command:
                - sleep
                - inf
              env:
                - name: USER_NAME
                  value: update-via-replacement
                - name: USER_PASSWORD
                  value: update-via-replacement
    

    And base/kustomization.yaml looks like:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    commonLabels:
      app: replacement-example
    
    resources:
      - deployment.yaml
    
    secretGenerator:
      - name: example
        literals:
          - password=secret
    
    configMapGenerator:
      - name: example
        literals:
          - username=alice
    

    So the base directory results in a Deployment, a Secret, and a ConfigMap. There are two overlays, env1 and env2. In both overlays I want to apply the same replacement configuration, so I put that into components/replace-username-password/kustomization.yaml:

    apiVersion: kustomize.config.k8s.io/v1alpha1
    kind: Component
    
    replacements:
      - source:
          kind: ConfigMap
          name: example
          fieldPath: data.username
        targets:
          - select:
              kind: Deployment
              name: example
            fieldPaths:
              - spec.template.spec.containers.[name=example].env.[name=USER_NAME].value
      - source:
          kind: Secret
          name: example
          fieldPath: data.password
        targets:
          - select:
              kind: Deployment
              name: example
            fieldPaths:
              - spec.template.spec.containers.[name=example].env.[name=USER_PASSWORD].value
    

    Now in overlays/env1/kustomization.yaml I can make use of this component:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    commonLabels:
      envName: env1
    
    resources:
      - ../../base
    
    components:
      - ../../components/replace-username-password
    

    And the same in overlays/env2:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    commonLabels:
      envName: env2
    
    resources:
      - ../../base
    
    components:
      - ../../components/replace-username-password