kuberneteskustomize

How to duplicate a specific object in a specific overlay from base with kustomize?


I have a base and two overlays: dev and prod, and all objects(Deployments, StatefulSets, Services etc.) defined in base layer are inherited in the two overlays. However, for some technical reason, I'd like to duplicate a specific Service "foo" in overlay prod to "foo-a", "foo-b", "foo-c", but keep it unchanged in overlay dev. Like the following:

.
├── base
│   ├── Service:foo
│   ├── Deployment:bar
│   ├── StatefulSet:baz
│   └── ...
└── overlays
    ├── dev
    │   ├── Service:foo      <-- base/Service:foo kept unchanged
    │   ├── Deployment:bar
    │   ├── StatefulSet:baz
    │   └── ...
    └── prod
        ├── Service:foo-a    <-- duplicated from base/Service:foo
        ├── Service:foo-b    <-- duplicated from base/Service:foo
        ├── Service:foo-c    <-- duplicated from base/Service:foo
        ├── Deployment:bar
        ├── StatefulSet:baz
        └── ...

Currently, the multibases solution is the closest solution, however, this solution duplicates all objects in a layer, but not a specific one.

I think I might use a script to achieve this, but I prefer the "kustomize" way. Any ideas? Thanks.


Solution

  • You can't do what you want with your existing layout; Kustomize is only able to include directories, not individual files, and there's no facility for "duplicating" a resource. If we move your Service manifest into a component and modify your overlays a bit, we can get closer.

    I ended up with the following layout (I didn't include your example statefuleset because it wasn't relevant to the answer):

    .
    ├── base
    │   ├── deployment.yaml
    │   └── kustomization.yaml
    ├── components
    │   └── service
    │       ├── kustomization.yaml
    │       └── service.yaml
    └── overlays
        ├── dev
        │   └── kustomization.yaml
        └── prod
            ├── kustomization.yaml
            ├── service-a
            │   └── kustomization.yaml
            ├── service-b
            │   └── kustomization.yaml
            └── service-c
                └── kustomization.yaml
    

    base/kustomization.yaml looks like this:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    resources:
    - deployment.yaml
    

    And deployment.yaml looks like:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: example
    spec:
      template:
        spec:
          containers:
          - name: example
            image: docker.io/traefik/whoami:latest
            env:
            - name: WHOAMI_PORT_NUMBER
              value: "8080"
            ports:
            - containerPort: 8080
              name: http
    

    Note that here and elsewhere I'm relying on kustomize to fill in labels and selectors.

    components/service/kustomization.yaml looks like this (note the apiVersion):

    apiVersion: kustomize.config.k8s.io/v1alpha1
    kind: Component
    
    resources:
    - service.yaml
    

    And service.yaml looks like this:

    apiVersion: v1
    kind: Service
    metadata:
      name: example
    spec:
      ports:
      - name: http
        port: 8080
    

    Then we create overlays/dev/kustomization.yaml like this:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    commonLabels:
      app: example
      environment: dev
    resources:
    - ../../base
    
    components:
    - ../../components/service/
    

    Running kustomize build overlays/dev gets us:

    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: example
        environment: dev
      name: example
    spec:
      ports:
      - name: http
        port: 8080
      selector:
        app: example
        environment: dev
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: example
        environment: dev
      name: example
    spec:
      selector:
        matchLabels:
          app: example
          environment: dev
      template:
        metadata:
          labels:
            app: example
            environment: dev
        spec:
          containers:
          - env:
            - name: WHOAMI_PORT_NUMBER
              value: "8080"
            image: docker.io/traefik/whoami:latest
            name: example
            ports:
            - containerPort: 8080
              name: http
    

    overlays/prod is a little more interesting. overlays/prod/kustomization.yaml looks like this:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    commonLabels:
      app: example
      environment: prod
    resources:
    - ../../base
    - service-a
    - service-b
    - service-c
    

    overlays/prod/service-a/kustomization.yaml looks like this:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    nameSuffix: -a
    
    components:
    - ../../../components/service/
    

    service-b and service-c are similar; we just use a different nameSuffix. Running kustomize build overlays/prod results in a Deployment and three Services:

    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: example
        environment: prod
      name: example-a
    spec:
      ports:
      - name: http
        port: 8080
      selector:
        app: example
        environment: prod
    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: example
        environment: prod
      name: example-b
    spec:
      ports:
      - name: http
        port: 8080
      selector:
        app: example
        environment: prod
    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: example
        environment: prod
      name: example-c
    spec:
      ports:
      - name: http
        port: 8080
      selector:
        app: example
        environment: prod
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: example
        environment: prod
      name: example
    spec:
      selector:
        matchLabels:
          app: example
          environment: prod
      template:
        metadata:
          labels:
            app: example
            environment: prod
        spec:
          containers:
          - env:
            - name: WHOAMI_PORT_NUMBER
              value: "8080"
            image: docker.io/traefik/whoami:latest
            name: example
            ports:
            - containerPort: 8080
              name: http
    

    I think that's roughly what you were looking for.