jsonmergejq

in jq, how to merge an array of object into 1 object with an array of values


Let's say I have this json:

[
  {
    "first": 12355,
    "second": "abc"
  },
  {
    "first": 89010,
    "second": "def"
  },
  {
    "first": 23423,
    "second": "hij"
  },
  {
    "first": 23456,
    "second": "klm"
  },
  {
    "first": 11111,
    "second": "nop"
  }
]

And I would like (a generic form) that merges them into 1 object where the value of each key is merged into an array of the corresponding values:

{
    "first" : [12355,89010,23423,23456,11111],
    "second" : ["abc","def","hij","klm","nop"]
}

I was trying this, but it produces no output at all.

reduce .[] as $final (
    {};
    ((. | keys) as $k |
        map(
            ( (.[$k] // []) += ($final[$k] // []))
        )
    )
)

Solution

  • I'd use to_entries (to access keys and values) as part of the iteration:

    reduce (.[] | to_entries[]) as $e ({}; .[$e.key] += [$e.value])
    

    Demo

    Here's another way using group_by to bring the matching items together:

    map(to_entries[]) | group_by(.key) | map({(first.key): map(.value)}) | add
    

    Demo

    Output:

    {
      "first": [
        12355,
        89010,
        23423,
        23456,
        11111
      ],
      "second": [
        "abc",
        "def",
        "hij",
        "klm",
        "nop"
      ]
    }
    

    Note: These solutions expect all input objects to have the same set of keys. Missing keys in single objects will lead to shortened, thus potentially misaligned output arrays. To readjust the arrays by pitching in null values, normalize the objects before destructuring them into entries by reconstructing them using a pre-defined set of keys, e.g. {first, second}:

    reduce (.[] | {first, second} | to_entries[]) as $e …
    # or
    map({first, second} | to_entries[]) | group_by(.key) …
    

    If, however, the actual set of keys is dynamic or unknown, screen and collect them first, e.g. using (map(keys[]) | unique) as $keys, then normalize based on that set using pick(.[$keys[]]) (pick was introduced with jq 1.7; older versions could use .[$keys[]] //= null instead):

    (map(keys[]) | unique) as $keys
    | reduce (.[] | pick(.[$keys[]]) | to_entries[]) as $e …
    # or
    (map(keys[]) | unique) as $keys
    | map(pick(.[$keys[]]) | to_entries[]) | group_by(.key) …