jsonjqreduce

JQ Cannot iterate over null


I am trying to create a jq that is dynamic in returning data. User Story: I have data with many fields and not named like I want. I want to be able to based on the data and schema transform the data.

Example: JSON

  "inputArray": [
    {
      "objective_id": "ac-1_obj.a-1",
      "description": "an access control policy is developed and documented;",
      "framework_id": 1,
      "domain_id": "ac",
      "domaintitle": "Access Control",
      "id": 0,
      "control_id": "ac-1",
      "controltitle": "Policy and Procedures"
    },
    {
      "objective_id": "ac-1_obj.a-2",
      "description": "the access control policy is disseminated to {{ insert: param, ac-01_odp.01 }};",
      "framework_id": 1,
      "domain_id": "ac",
      "domaintitle": "Access Control",
      "id": 0,
      "control_id": "ac-1",
      "controltitle": "Policy and Procedures"
    },
    {
      "objective_id": "ac-1_obj.a-3",
      "description": "access control procedures to facilitate the implementation of the access control policy and associated controls are developed and documented;",
      "framework_id": 1,
      "domain_id": "ac",
      "domaintitle": "Access Control",
      "id": 0,
      "control_id": "ac-1",
      "controltitle": "Policy and Procedures"
    }
  ],
  "schema": [
    { "new": "objective", "input": "objective_id" },
    { "new": "framework", "input": "framework_id" },
    { "new": "control", "input": "control_id" },
    { "new": "title", "input": "controltitle" },
    { "new": "description", "input": "description" }
  ]
}

JQ

  inputArray: [
    .inputArray[] |
    reduce (.schema[] | {new: .new, input: .input}) as $map (
      {}; 
      .[$map.new] = .[$map.input]
    )
  ]
}

Error

jq: error (at /tmp/data.json:40): Cannot iterate over null (null) undefined

Solution

  • The error you are encountering is related to referencing the top level schema object while iterating through the contents of inputArray without having a proper reference back to it.

    So, you need to be able to get the value of the current item with the key (item[ that maps to the key in the schema.

    Try this:

    {
      transformedArray: (
        [.inputArray[] as $item |
          reduce .schema[] as $s (
            {}; 
            . + { ($s.new): $item[$s.input] }
          )
        ]
      )
    }
    

    $item is the current item being iterated over in inputArray.
    $s is the schema, then $s.new is the new key that replaces the old key $s.input.
    The value is then obtained with $item[$s.input]

    enter image description here