jsonjq

How to exclude the elements from one array that exist in another array by some of their properties using jq?


Assuming two json files: a.json

[
  {
    "category": "utils",
    "name": "yq"
  },
  {
    "category": "utils",
    "name": "yq4"
  },
  {
    "category": "utils",
    "name": "yip"
  },
  {
    "category": "utils",
    "name": "jq"
  },
  {
    "category": "utils",
    "name": "yip-systemd"
  },
  {
    "category": "utils",
    "name": "system-tools"
  }
]

b.json

[
  {
    "category": "utils",
    "name": "jq"
  },
  {
    "category": "utils",
    "name": "yq"
  },
  {
    "category": "utils",
    "name": "yip"
  },
  {
    "category": "utils",
    "name": "yq4"
  },
  {
    "category": "utils",
    "name": "yip-systemd"
  },
  {
    "category": "utils",
    "name": "system-tools"
  },
  {
    "category": "utils",
    "name": "test"
  }
]

In order to get the objects in b that are not in a, can run:

jq -n --argfile a a.json --argfile b b.json '$b-$a | .[] | .category + "/" + .name'

the result being:

"utils/test"

Using jq, how to manage the situation when additional properties are set, having different values"?

For example: a.json

[
  {
    "category": "utils",
    "name": "yq",
    "version": "1"
  },
  {
    "category": "utils",
    "name": "yq4",
    "version": "1"
  },
  {
    "category": "utils",
    "name": "yip",
    "version": "1"
  },
  {
    "category": "utils",
    "name": "jq",
    "version": "1"
  },
  {
    "category": "utils",
    "name": "yip-systemd",
    "version": "1"
  },
  {
    "category": "utils",
    "name": "system-tools",
    "version": "1"
  }
]

b.json

[
  {
    "category": "utils",
    "name": "jq",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "yq",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "yip",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "yq4",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "yip-systemd",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "system-tools",
    "version": "2"
  },
  {
    "category": "utils",
    "name": "test",
    "version": "2"
  }
]

having the result:

"utils/test"

Solution

  • A very simple solution would be to construct the strings before subtraction:

    jq -n --argfile a a.json --argfile b b.json '[$a, $b | map(.category + "/" + .name)] | (last - first)[]'
    
    "utils/test"
    

    However note that --argfile is deprecated. To get rid of it, you could use inputs while providing the files as regular input files:

    jq -n '[inputs | map(.category + "/" + .name)] | (last - first)[]' a.json b.json
    
    "utils/test"
    

    Demo

    Here's another take deleting keys from an INDEX, which can be fed a filter criterion (taking your string construction from earlier):

    jq 'def fc: .category + "/" + .name; INDEX(fc) | del(.[input[] | fc]) | keys[]' b.json a.json
    
    "utils/test"
    

    Demo


    Is there any way to write it in a manner that would explicitly compare the properties as a filter does, like a.name != b.name and a.category != b.category?

    Do you mean something along these lines?

    jq 'reduce .[] as $a (input; map(select(
      $a.name != .name or $a.category != .category
    )))' a.json b.json
    
    [
      {
        "category": "utils",
        "name": "test",
        "version": "2"
      }
    ]
    

    Demo