jsonmergejqinverse

jq: how to perform unmerge / multi-level object subtraction i.e. given X and Y, find Z such that X * Z = Y


Using jq we can easily merge two multi-level objects X and Y using *:

X='{
    "a": 1,
    "b": 5,
    "c": {
      "a": 3
    }
  }' Y='{
    "d": 2,
    "a": 3,
    "c": {
      "x": 10,
      "y": 11
    }
  }' && Z=`echo "[$X,$Y]"|jq '.[0] * .[1]'` && echo "Z='$Z'"

gives us:

Z='{
  "a": 3,
  "b": 5,
  "c": {
    "a": 3,
    "x": 10,
    "y": 11
  },
  "d": 2
}'

But in my case, I'm starting with X and Z and want to calculate Y (such that X * Y = Z). If we only have objects with scalar properties, then jq X + Y equals Z, and we can also calculate Y as jq Z - X. However, this fails if X or Y contain properties with object values such as in the above example:

X='{
    "a": 1,
    "b": 5,
    "c": {
      "a": 3
    }
  }' Z='{
  "a": 3,
  "b": 5,
  "c": {
    "a": 3,
    "x": 10,
    "y": 11
  },
  "d": 2
}' && echo "[$X,$Z]" | jq '.[1] - .[0]'

throws an error jq: error (at <stdin>:16): object ({"a":3,"b":...) and object ({"a":1,"b":...) cannot be subtracted

Is there an elegant solution to this problem with jq?

UPDATE: I've accepted the answer that I found easier to read / maintain and with superior performance. In addition, I found a wrinkle in my need which was that if X contained a key K that was not present in Z, I needed the output (Y) to nullify it by containing the key K with a value of null.

The best way I could come up with to do this was to pre-process Z to add the missing keys using the below:

def add_null($y):
    reduce (to_entries[] | [ .key, .value ] ) as [ $k, $v ] (
       $y;
       if $y | has($k) | not then
          .[$k] = null
       elif $v | type == "object" then
          .[$k] = ($v | add_null($y[$k]))
       else
          .[$k] = $v
       end
    );

so we end up with:

def add_null(...);

def remove(...);

. as [ $X, $Z ] | ($X | add_null($Z)) | remove($X)

Any better suggestions to this variation are still appreciated!


Solution

  • def remove($o2):
       reduce ( to_entries[] | [ .key, .value ] ) as [ $k, $v1 ] (
          {};
          if $o2 | has($k) | not then
             # Keep existing value if $o2 doesn't have the key.
             .[$k] = $v1
          else
             $o2[$k] as $v2 |
             if $v1 | type == "object" then
                # We're comparing objects.
                ( $v1 | remove($o2[$k]) ) as $v_diff |
                if $v_diff | length == 0 then
                   # Discard identical values.
                   .
                else
                   # Keep the differences of the values.
                   .[$k] = $v_diff
                end
             else
                # We're comparing non-objects.
                if $v1 == $v2 then
                   # Discard identical values.
                   .
                else
                   # Keep existing value if different.
                   .[$k] = $v1
                end
             end
          end
       );
    
    . as [ $Z, $X ] | $Z | remove($X)
    

    Demo on jqplay

    or

    def sub($v2):
       (       type ) as $t1 |
       ( $v2 | type ) as $t2 |
       if $t1 == $t2 then
          if $t1 == "object" then
             with_entries(
                .key as $k |
                .value = (
                   .value |
                   if $v2 | has($k) then sub( $v2[$k] ) else . end
                )
             ) |
             select( length != 0 )
          else
             select( . != $v2 )
          end
       else
          .
       end;
    
    . as [ $Z, $X ] | $Z | sub($X)
    

    Demo on jqplay