jsonjqpgbackrest

how to get a max per group and subgroup using jq


jq is an amazing tool and it does a lot. as input I have

[
  {
    "backup": [
      {
        "timestamp": { "start": 1642144383, "stop": 1642144386 },
        "info": {  "size": 1200934840},
        "type": "full"
      },
      {
        "timestamp": {"start": 1642144388, "stop":  1642144392 },
        "info": { "size": 1168586300
        },
        "type": "incr"
      },
      {
        "timestamp": {"start": 1642145388, "stop":  1642145392 },
        "info": { "size": 1168586330
        },
        "type": "incr"
      }
    ],
    "name": "dbname1"
  },
  {
    "backup": [
      {
        "timestamp": { "start": 1642144383, "stop": 1642144386 },
        "info": {  "size": 1200934840},
        "type": "full"
      },
      {
        "timestamp": {"start": 1642144388, "stop":  1642144392 },
        "info": { "size": 1168586300
        },
        "type": "incr"
      }
    ],
    "name": "dbname2"
  }
]

and using

jq 'map([.backup[] + {name}] | max_by(.timestamp.stop))'

I get the latest timestamp.stop for a name. How should I change this to get the latest timestamp.stop for a name and group? in SQL this would be something like max(.timestamp.stop) group by .name,.type Hoping for output like:

[
  {
    "timestamp": {
      "start": 1642144383,
      "stop": 1642144386
    },
    "info": {
      "size": 1200934840
    },
    "type": "full",
    "name": "dbname1"
  },
  {
    "timestamp": {
      "start": 1642145388,
      "stop": 1642145392
    },
    "info": {
      "size": 1168586330
    },
    "type": "incr",
    "name": "dbname1"
  },
  {
    "timestamp": {
      "start": 1642144383,
      "stop": 1642144386
    },
    "info": {
      "size": 1200934840
    },
    "type": "full",
    "name": "dbname2"
  },
  {
    "timestamp": {
      "start": 1642144388,
      "stop": 1642144392
    },
    "info": {
      "size": 1168586300
    },
    "type": "incr",
    "name": "dbname2"
  }
]

Solution

  • Remove the inner brackets to flatten the array, then group_by both criteria (which makes your criteria an array), and map your max_by onto the result array:

    jq 'map(.backup[] + {name}) | group_by([.name, .type]) | map(max_by(.timestamp.stop))'
    
    [
      {
        "timestamp": {
          "start": 1642144383,
          "stop": 1642144386
        },
        "info": {
          "size": 1200934840
        },
        "type": "full",
        "name": "dbname1"
      },
      {
        "timestamp": {
          "start": 1642145388,
          "stop": 1642145392
        },
        "info": {
          "size": 1168586330
        },
        "type": "incr",
        "name": "dbname1"
      },
      {
        "timestamp": {
          "start": 1642144383,
          "stop": 1642144386
        },
        "info": {
          "size": 1200934840
        },
        "type": "full",
        "name": "dbname2"
      },
      {
        "timestamp": {
          "start": 1642144388,
          "stop": 1642144392
        },
        "info": {
          "size": 1168586300
        },
        "type": "incr",
        "name": "dbname2"
      }
    ]
    

    Demo