amazon-web-servicesshelljq

Extract data and sibling node from sub-array


I am trying to parse the output of AWS CLI describe-volumes - the raw output looks something like this:

{
    "Volumes": [
        {
            "Attachments": [
                {
                    "Device": "/dev/xv1d",
                    "InstanceId": "i-abcdef1234",
                    "State": "attached",
                    "VolumeId": "vol-1234abcdef",
                    "DeleteOnTermination": false
                }
            ],
            "AvailabilityZone": "us-east-1a",
            "Size": 500,
            "VolumeId": "vol-1234abcdef",
            "Tags": [
                {
                    "Key": "version",
                    "Value": "1234"
                },
                {
                    "Key": "VolName",
                    "Value": "myhostname-example-com-data"
                }
            ],
            "VolumeType": "st1",
        },
        ....

I want to get the VolumeId, VolumeType, Size, Iops and the Tags.Value but only where Tags.Key is "VolName".

The command below gives me most of what I want:

jq '.Volumes[] | [.VolumeId, .VolumeType, .Size, .Iops, .VolumeType]' <described.json

[
  "vol-1234abcdef",
  "st1",
  500,
  null,
  "st1"
],
...

But if I use the command below to add on Tag value, I only get the tag values returned - not the other attributes:

jq '.Volumes[] | [.VolumeId, .VolumeType, .Size, .Iops, .VolumeType, .Tags[] | select(.. | .Key? =="Name").Value]' <described.json

[
   "myhostname-example-com-data"
],
...

How can I get all the desired attributes as a single array element? I don't mind if that is an array of objects or an array of arrays as per the first example.

(Tags may contain additional elements which I am not interested in - I can't exclude where Tags.Key is version)


Solution

  • You need to use parentheses to force evaluation order. , has higher precedence than |; so your code is equivalent to … | [ (.VolumeId, .VolumeType, …) | select(…).Value ].

    .Volumes[]
    | [
        .VolumeId,
        .VolumeType,
        .Size,
        .Iops,
        .VolumeType,
        (.Tags[] | select(.. | .Key? == "VolName").Value)
    ]
    

    Since your Tags value comprises key-value pairs, you can use the very handy from_entries filter:

    .Volumes[]
    | [
        .VolumeId,
        .VolumeType,
        .Size,
        .Iops,
        .VolumeType,
        (.Tags | from_entries.VolName)
    ]
    

    Output:

    [
      "vol-1234abcdef",
      "st1",
      500,
      null,
      "st1",
      "myhostname-example-com-data"
    ]
    

    Alternatively, map to an array of objects:

    .Volumes
    | map({
        VolumeId,
        VolumeType,
        Size,
        Iops,
        VolumeType,
        Tags: .Tags | from_entries.VolName
    })
    

    Output:

    [
      {
        "VolumeId": "vol-1234abcdef",
        "VolumeType": "st1",
        "Size": 500,
        "Iops": null,
        "Tags": "myhostname-example-com-data"
      }
    ]