bashparsingyamlyq

How do I retrieve an attribute's keys inside a .yaml file using YQ?

I have the following yaml:

attr_keys:
  x:
    y: [1, 2, 3]
    z: 7

I need to create a bash script that parses the yaml file, creates various combinations of all the keys' values found under attr_keys.x and runs a script various times, passing each time, a different combination of parameters.

Now I am having problems parsing the yaml file. When there is only one key under attr_keys.x there are no issues, the script works fine. However when there are multiple keys, it breaks.

I am using yq to parse the file:

parameters_keys=$(yq e ".attr_keys.x | keys | .[] " config.yaml)
for parameter_key in "${parameters_keys[@]}"
      do
        echo "Key: $parameter_key"
      done

If we were to consider the following yaml:

attr_keys:
  x:
    y: [1, 2, 3]

The above script prints Key: y and this is correct. Following this behaviour, if I were to consider the following yaml file:

attr_keys:
  x:
    y: [1, 2, 3]
    z: 7

The expected output should be

Key: y
Key: z

However I get:

Key: y
z

Apparently, multiple keys are parsed as a whole string and considered as one big "key". What I need is to handle multiple keys as an array or a list or something similar, any suggestions on how to do so?

I am using yq (https://github.com/mikefarah/yq/) version v4.34.1

I tried chaining the bash command:

parameters_keys=$(yq e ".attr_keys.x | keys | .[] " config.yaml)

to

parameters_keys=$(yq e ".attr_keys.x | keys " config.yaml)

and to

parameters_keys=$(yq e ".attr_keys.x | keys | [.[]] " config.yaml)

but none of them worked.

Solution

  • There is no issue with yq, the issue is with your Bash script. varname=$(…) will always be a (possibly multiline) string, regardless of the logic of the producer. Thus, you cannot just transfer the array logic into Bash, you have to create the array yourself from within Bash, e.g. by assuming item boundaries at line breaks. Use parens for that:

    parameters_keys=($(yq e ".attr_keys.x | keys | .[]" config.yaml))

    However, it's safer to use one of the Bash builtins (e.g. readarray) to do that:

    readarray -t parameters_keys < <(yq e ".attr_keys.x | keys | .[]" config.yaml)