jsonansibleuntil-loop

recursive parsing in json and get the value using ansible


I need to get the value of key in json from ansible play. Here is the issue is - Not sure the key will be in particular level of Json. It may vary from json to json. So, I need to 'serach' for the keys in json blindly and get the values for them [Not sure how many keys are avalible and how deep they are available]. Here I'm giving the sample json and play to achieve same.

JSON Data:

{
    "status": "SUCCESS",
    "errors": [
        {
            "jobname": "DbcAllJob_td_part_20190211150403_1",
            "errorList": [
                {
                    "id": "1",
                    "migrationId": "b9e19f6b-bc2e-4e2b-8fe9-20efb74d8289",
                    "jobId": "27",
                    "backupPhase": "DATA",
                    "restorePhase": "NULL",
                    "errorType": "ABORT",
                    "atTimeStamp": "2019/02/11 15:27:46",
                    "errorStatus": "UNRESOLVED",
                    "errorcode": "1148",
                    "errortext": "1148: DSA DSMAIN-BARNC Communication Exception: 1001:(**********K) recv() call has EOF",
                    "report": {
                        "root": {
                            "system": "TARGET",
                            "host": "**********",
                            "state_text": "2019/02/11 07:26:52.181 [      27:29152:29152] ERROR Executor - Exception received by master thread: recv() call has EOF",
                            "entityId": "**********_clienthandler",
                            "logTime": "2019/02/11 07:26:52.181",
                            "peers": [
                                "**********_clienthandler"
                            ],
                            "phase": "",
                            "rootError": "BarNC - User shutdown request from command line received, stopping BARNC",
                            "software": "clienthandler"
                        },
                        "sequence": [
                            {
                                "host": "sdt09586",
                                "state_text": "Feb 11 07:25:20 sdt09586 DSA[27578]: CRITICAL: DSA: 4751002 #CRITICAL: DataStreamController: 4751002 #DSC version[16.20.24.00], Commons version[16.20.24.00], DSC  cannot connect to activeMQ Broker.",
                                "entityId": "sdt09586_messages",
                                "logTime": "2019/02/11 7:25:20.000",
                                "peers": [],
                                "phase": "",
                                "software": "messages"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "Feb 11 07:26:52 ********** Teradata[19593]: INFO: Teradata: 6670 # DSMAIN 1148: DSA DSMAIN-BARNC Communication Exception: 1001:(**********K) recv() call has EOF",
                                "entityId": "**********_messages",
                                "logTime": "2019/02/11 7:26:52.000",
                                "peers": [],
                                "phase": "",
                                "software": "messages"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "Feb 11 07:26:52 ********** Teradata[6369]: INFO: Teradata: 3265 #Transaction has been Aborted by Administrator or Operations Staff.",
                                "entityId": "**********_messages",
                                "logTime": "2019/02/11 7:26:52.000",
                                "peers": [],
                                "phase": "",
                                "software": "messages"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "2019/02/11 07:26:52.900 (134226672): Exception rethrown caught in DSMBackupTask::BeginWriteTask.",
                                "entityId": "27_**********_barlog_26623_34_3_3",
                                "logTime": "2019/02/11 07:26:52.900",
                                "peers": [
                                    "AMP =1"
                                ],
                                "phase": "",
                                "software": "barlog_26623_34_3_3"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "2019/02/11 07:26:52.900 (134226672): SendInternalAbort assumeMsgFromStpAab = 1",
                                "entityId": "27_**********_barlog_26623_34_3_3",
                                "logTime": "2019/02/11 07:26:52.900",
                                "peers": [
                                    "AMP =1"
                                ],
                                "phase": "",
                                "software": "barlog_26623_34_3_3"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "2019/02/11 07:26:52.900 (134226672): SendInternalAbort, partition = 34, jobId = 27",
                                "entityId": "27_**********_barlog_26623_34_3_3",
                                "logTime": "2019/02/11 07:26:52.900",
                                "peers": [
                                    "AMP =1"
                                ],
                                "phase": "",
                                "software": "barlog_26623_34_3_3"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "2019/02/11 07:26:52.900 (134226672): errorCode = 1148, errorTextLength = 84, errorText = 1148: DSA DSMAIN-BARNC Communication Exception: 1001:(**********K) recv() call has EOF",
                                "entityId": "27_**********_barlog_26623_34_3_3",
                                "logTime": "2019/02/11 07:26:52.900",
                                "peers": [
                                    "AMP =1"
                                ],
                                "phase": "",
                                "software": "barlog_26623_34_3_3"
                            },
                            {
                                "system": "SOURCE",
                                "host": "**********",
                                "state_text": "2019/02/11 07:26:52.910 (134254672): AbortTask has a message...",
                                "entityId": "27_**********_barlog_26623_34",
                                "logTime": "2019/02/11 07:26:52.910",
                                "peers": [],
                                "phase": "DATA/BUILD",
                                "software": "barlog_26623_34"
                            }
                        ]
                    },
                    "remedy": [
                        {
                            "seqNum": 1,
                            "remedy": "Configure all machines to have same bynet configuration",
                            "remedyOpcode": "RECONFIGURE_BYNET",
                            "remedyInfo": [
                                {
                                    "systemName": "**********",
                                    "ipAddress": "**********",
                                    "username": "**********",
                                    "password": "**********",
                                    "osUser": "root"
                                },
                                {
                                    "systemName": "**********",
                                    "ipAddress": "**********",
                                    "username": "**********",
                                    "password": "**********",
                                    "osUser": "root"
                                }
                            ]
                        },
                        {
                            "seqNum": 2,
                            "remedy": "Bring ClientHandler up on **********",
                            "remedyOpcode": "RESTART_SERVICE",
                            "remedyInfo": {
                                "systemName": "**********",
                                "ipAddress": "**********",
                                "username": "**********",
                                "password": "**********",
                                "osUser": "root"
                            }
                        },
                        {
                            "seqNum": 3,
                            "remedy": "Re-submit the job",
                            "remedyOpcode": "RESUBMIT",
                            "remedyInfo": null
                        }
                    ],
                    "detailedreport": [
                        {
                            "component": "TPANode : **********",
                            "description": "TPANODE status : Up and Running",
                            "additionalProperties": {}
                        },
                        {
                            "component": "TPANode : **********",
                            "description": "TPANODE status : Up and Running",
                            "additionalProperties": {}
                        },
                        {
                            "component": "Bynet : [**********, **********]",
                            "description": "Bynet configuration mismatch",
                            "additionalProperties": {}
                        },
                        {
                            "component": "ClientHandler : **********",
                            "description": "ClientHandler status : not running",
                            "additionalProperties": {}
                        },
                        {
                            "component": "ActiveMQ : **********",
                            "description": "ActiveMQ status : running",
                            "additionalProperties": {}
                        },
                        {
                            "component": "ActiveMQ Queue: **********",
                            "description": "ActiveMQ message count : 13",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DSC : **********",
                            "description": "DSC status : running, DSARest status : running",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DSC Processes: **********",
                            "description": "Number of DSC processes running : 1",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DSMAIN : **********",
                            "description": "DSMain status : Up and Running, Left over Processes : No left over processes",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DSMAIN : **********",
                            "description": "DSMain status : Up and Running, Left over Processes : No left over processes",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DatabaseConnection : **********",
                            "description": "Database status : UP",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DatabaseConnection : **********",
                            "description": "Database status : UP",
                            "additionalProperties": {}
                        },
                        {
                            "component": "DatabaseConnection : **********",
                            "description": "Database status : UP",
                            "additionalProperties": {}
                        }
                    ]
                },
                {
                    "id": "2",
                    "migrationId": "b9e19f6b-bc2e-4e2b-8fe9-20efb74d8289",
                    "jobId": "27",
                    "backupPhase": null,
                    "restorePhase": null,
                    "errorType": "ABORT",
                    "atTimeStamp": "",
                    "errorStatus": null,
                    "errorcode": "1160",
                    "errortext": "An error occurred restoring statistics.",
                    "report": {
                        "root": null,
                        "sequence": null
                    },
                    "remedy": [
                        {
                            "seqNum": -1,
                            "remedy": null,
                            "remedyOpcode": null,
                            "remedyInfo": null
                        }
                    ],
                    "detailedreport": [
                        {
                            "component": null,
                            "description": null,
                            "additionalProperties": {}
                        }
                    ]
                }
            ]
        }
    ]
}

From above json I need to read the 'remedy' values. But not sure at what level they are available always.

Here is the ansible script for the same.

---
- hosts: localhost
  tasks:

  - name: loop
    set_fact:
       migration_detail_status: "{{ lookup('file', 'inj_error.json') }}"
    until: "'{{ migration_detail_status.errors.0.errorList.0.remedy.1.remedy | search('Bring ClientHandler up on') }}' and '{{ migration_detail_status.errors.0.errorList.0.remedy.1.remedyOpcode | search('RESTART_SERVICE') }}'"
    retries:  2
    delay: 5
    ignore_errors: yes

  - debug:
        var: migration_detail_status.errors.0.errorList.0.remedy.1.remedyOpcode

At until condition I've traversed over json as I know the format of json. but, it may not be same for all the cases. So, please help to traverse to get the searching key:value from json irrespective of their level.


Solution

  • You can get a list of all remedy's with JMESPath query: errors[].errorList[].remedy[].remedy.

    [
      "Configure all machines to have same bynet configuration",
      "Bring ClientHandler up on **********",
      "Re-submit the job"
    ]
    

    In Ansible you would use json_query filter to apply JMESPath queries.

    If you also need opcodes: errors[].errorList[].remedy[].{r:remedy,o:remedyOpcode}

    [
      {
        "r": "Configure all machines to have same bynet configuration",
        "o": "RECONFIGURE_BYNET"
      },
      {
        "r": "Bring ClientHandler up on **********",
        "o": "RESTART_SERVICE"
      },
      {
        "r": "Re-submit the job",
        "o": "RESUBMIT"
      },
      {
        "r": null,
        "o": null
      }
    ]
    

    Example:

    - debug:
        msg: "{{ migration_detail_status | json_query('errors[].errorList[].remedy[].remedy') }}"