[SOLVED] Change value in yaml file without changing the files structure

Change value in yaml file without changing the files structure

I'm thinking about setting up a GitOps CI/CD pipeline in my cluster with Jenkins and ArgoCD.

For a start I want to have a repository for my CI environment with some values files for the Helm charts of my applications.

One thing that I cannot really figure out is, how I can automatically edit the Helm values files without changing the whole structure of the files.

The Jenkins pipeline methods for reading and writing yaml files will fully recreate it and in the process re-format the whole file.

yq does not (seem to) re-order the keys, but it removes empty lines and comments, for example.

Only other thing that comes to my mind is using sed. But that feels kind-of wrong. And it might easily break. Like when I add a second key in another group with the same name. Or add a or remove keys.

Here's an example, to make it a bit more clear:

I have two repositories, one for my application, one for my CI, NI, ... configuration.

My application repo is not that important. Just some application. My config repository looks somewhat like this:

 .
...
├── ci
│   ├── app1
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   ├── app2
...
├── staging
│   ├── app1
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   ├── app2
...

The values.yaml file of app1 in the ci folder might look like this:

app1:
  image:
    tag: 1.33.7

  ingress:
    enabled: true
    hosts:
      - app1.my-internal-ci.company.com
  ...

The actual helm charts are somewhere else and here is only the configuration for the the applications on the environments.

ArgoCD watches this repo and takes care that the desired state (as specified by the state of the repo) and the actual state in the cluster match (see https://argoproj.github.io/cd/ for more on the tool).

As a last step my CI pipeline for app1, that builds a docker image from the code in my application repo, should update the values.yaml file in the ci folder in the configuration repo with the new version 1.33.8 of the application that was just build and push it as a new commit to the main branch.

Besides that, there are other values configured, like the ingress for example, that I update manually in the repo, if needed.

Since the files are updated automatically by the CI build pipeline and manually by a developer / DevOps engineer, I would like to be able to keep them easily readable by a human (order of the keys, newlines, comments, ...)

Is there any other way of achieving this?

Thanks in advance!

Solution

After some further digging I think the best option I have right now is yq. In newer versions (> 3.0.0; https://github.com/mikefarah/yq/issues/19) it should not remove any comments anymore. And with the newline-thing I can live, I think.

So if you have the files as mentioned above you could update your image tag with:

yq -i '.app1.image.tag = "1.33.8"' ci/app1/values.yaml

Worth mentioning: If you're using kustomize instead of Helm, there is a built-in for this: kustomize edit set image my-app=myregistry.io/my-platform/my-app:1.2.3.

Thanks for your ideas and suggestions!