jsonshelljq

Replace specific keys in a deeply embedded json file


I have a json file (origin.json) generated locally, I'd like to replace some keys in this origin.json and generate a remote.json so that I could send it to a remote server following it's endpoint payload format.

My origin.json is large and deeply embedded, I could iterate each keys and replace those I need to. But I am wondering is there an efficient and fancy tool could do the same? Something like jq?

Below are my embedded json

{
        "timeoutMs": 3000,
        "requestTopic": "local-cron",
        "searchQuery": {
            "checkin": "2023-01-10",
            "checkout": "2023-01-11",
            "numberRoomsNeeded": 0,
            "adultsTotal": 2,
            "childrenTotal": 0,
            "currency": "EUR"
        },
        "requestContext": {
            "userId": 666666666,
            "userAuthLevel": 2,
            "isUserstar": true,
            "visitorCc1": "cn",
            "trafficSourceId": 0,
            "siteTypeId": 9,
            "detectedUserType": "normal",
            "travelPurpose": 2,
            "affiliateId": 12345,
            "languageCode": "en-us",
            "currency": "CNY",
            "siteType": 1,
            "serverRole": "cron",
            "action": "bp",
            "visitorIdentifier": [
                {
                    "type": "id-single",
                    "uvi": "00000000000000000000000"
                },
                {
                    "type": "user-identity",
                    "uvi": "66666666"
                },
                {
                    "type": "user",
                    "uvi": "77777777777"
                }
            ],
            "isInternal": true,
            "enableExperiments": true,
            "shouldTrackRequestInExperiments": true,
            "starSettings": {
                "isUserstar": true,
                "isUserstarControlGroup": false,
                "canStarUserSeeFreeBreakfast": true,
                "canStarUserSeeFreeRoomUpgrade": true,
                "starTier": 5,
                "topstarBenefit": "",
                "isRightstar": true,
                "starDynamicPricing": {
                    "canSeestarDynamicPricingLevel3": true
                },
                "canStarUserSeeFreeCleaningFee": true,
                "starVipSettings": [
                    {
                        "eligible": true,
                        "benefitName": "no_et",
                        "programType": "PriceMatchTrial",
                        "percentage": 0
                    }
                ]
            },
            "isCsRelocationMode": false,
            "tripValueContext": {},
            "visitorCountryRegion": "sh",
            "paymentTiming": 1,
            "includeConditional": false
        },
        "showDebug": false,
        "hits": [
            {
                "hhhhid": 8228082,
                "ppblock": {
                    "allMatchingBlocks": [
                        {
                            "blockId": 1,
                            "rawBlock": {
                                "occupancy": 2,
                                "price": 34425,
                                "roomId": 822808201,
                                "policygroupId": 346547507,
                                "mealplan": 2,
                                "channel": 581,
                                "currencyId": 2,
                                "maxPersons": 3,
                                "flags": 0,
                                "freeCancelUntil": 0,
                                "priceBase10Exponent": -2,
                                "packageId": 0,
                                "paymenttermsId": 38,
                                "vrFlags": 0,
                                "bundleId": 0
                            },
                            "blockStay": {
                                "stayNights": [
                                    {
                                        "polId": 346547507,
                                        "rateId": 25728208,
                                        "curId": 2,
                                        "price": 344.25,
                                        "price1": 0,
                                        "channelId": 581,
                                        "occupancy": 2,
                                        "roomId": 822808201,
                                        "initialPrice": 405,
                                        "initialPrice1": 0
                                    }
                                ],
                                "stayNrRooms": 1,
                                "stayAvailableUntil": 1956105,
                                "stayPrice": 344.25,
                                "stayFlashDeal": 0,
                                "stayPromoTextId": 0,
                                "stayMinAdvanceRes": 1673388000,
                                "stayInventorySegmentId": 0,
                                "stayExperimentFlags": 0,
                                "stayRoomRateFlags": 4,
                                "stayIncludedProducts": 0
                            }
                        }
                        ]
                    },
                    "selectedBlocks": [
                        "822808201_346547507_2_2_0"
                    ],
                    "selected": {
                        "822808201_346547507_2_2_0": 1
                    }
                }
            ],
            "pipeline": 3
}

Here I flagged several keys I'd like to replace with '==> (new key)'

jq '.. | keys?' star-dragongate.json 
[
  "hits",
  "pipeline",
  "requestContext",
  "requestTopic",
  "searchQuery",
  "showDebug", ==> showdebug 
  "timeoutMs"
]
[
  "adultsTotal",
  "checkin",
  "checkout",
  "childrenTotal",
  "currency",
  "numberRoomsNeeded"
]
[
  "action",
  "affiliateId", ==> Affilateid
  "currency",
  "detectedUserType",
  "enableExperiments",
  "starSettings",
  "includeConditional",
  "isCsRelocationMode",
  "isInternal",
  "isUserstar",
  "languageCode",
  "paymentTiming",
  "serverRole",
  "shouldTrackRequestInExperiments", ==> inexperiments
  "siteType",
  "siteTypeId",
  "trafficSourceId",
  "travelPurpose",
  "tripValueContext",
  "userAuthLevel",
  "userId",
  "visitorCc1",
  "visitorCountryRegion",
  "visitorIdentifier"
]
[
  0,
  1,
  2
]
[
  "type",
  "uvi"
]
[
  "type",
  "uvi"
]
[
  "type",
  "uvi"
]
[
  "canStarUserSeeFreeBreakfast",
  "canStarUserSeeFreeCleaningFee",
  "canStarUserSeeFreeRoomUpgrade",  ==> freeroom_upgrade
  "starDynamicPricing",
  "starTier",
  "starVipSettings",
  "isRightstar",
  "isUserstar",
  "isUserstarControlGroup",
  "topstarBenefit"
]
[
  "canSeestarDynamicPricingLevel3"
]
[
  0
]
[
  "benefitName",
  "eligible",
  "percentage",
  "programType"
]
[]
[
  0
]
[
  "hhhhid",
  "ppblock",
  "selected",
  "selectedBlocks"
]
[
  "allMatchingBlocks"
]
[
  0
]
[
  "blockId",
  "blockStay",
  "rawBlock"
]
[
  "bundleId",  ==> bundle_id
  "channel",
  "currencyId",
  "flags",
  "freeCancelUntil",
  "maxPersons",
  "mealplan",
  "occupancy",
  "packageId",
  "paymenttermsId",
  "policygroupId",
  "price",
  "priceBase10Exponent",
  "roomId",
  "vrFlags"
]
[
  "stayAvailableUntil",
  "stayExperimentFlags",
  "stayFlashDeal",
  "stayIncludedProducts",
  "stayInventorySegmentId",
  "stayMinAdvanceRes",
  "stayNights",
  "stayNrRooms",
  "stayPrice",
  "stayPromoTextId", ==> staypromotextid
  "stayRoomRateFlags"
]
[
  0
]
[
  "channelId",
  "curId",
  "initialPrice",
  "initialPrice1",
  "occupancy",
  "polId",
  "price",
  "price1",
  "rateId",
  "roomId"
]
[
  0
]
[
  "822808201_346547507_2_2_0"
]

The keys I need to replace located in different embed layer and blocks of this json.

Is there any suggestion on how to replace those keys in an efficient way ?

PS: The keys in json file are in static location, not dynamically change all the time.


Solution

  • To rename a field name, you could use with_entries, which gives you access to each .key. Reset it by assignment.

    Now, what's still unclear is how you want to find the keys in question (programmatically). If their locations are static, and you know them, address them individually, as in:

    .requestContext.starSettings |= with_entries((.key | select(. == "canStarUserSeeFreeRoomUpgrade")) = "freeroom_upgrade")
    | .hits[].ppblock.allMatchingBlocks[].rawBlock |= with_entries((.key | select(. == "bundleId")) = "bundle_id")
    # and so on...
    

    To reduce redundant code, you could also move the renaming portion of it into its own function, and then just call that, e.g.:

    def rename($old; $new):
      with_entries((.key | select(. == $old)) = $new);
    
    .requestContext.starSettings |= rename("canStarUserSeeFreeRoomUpgrade"; "freeroom_upgrade")
    | .hits[].ppblock.allMatchingBlocks[].rawBlock |= rename("bundleId";  "bundle_id")
    # and so on...
    

    Or move the location also into the function, e.g.:

    def rename_at(path; $old; $new):
      path |= with_entries((.key | select(. == $old)) = $new);
    
    rename_at(.requestContext.starSettings; "canStarUserSeeFreeRoomUpgrade"; "freeroom_upgrade")
    | rename_at(.hits[].ppblock.allMatchingBlocks[].rawBlock; "bundleId";  "bundle_id")
    # and so on...
    

    If their location is unknown, and you want to replace them just based on their (local) name, you need to traverse the document, and check if you hit a matching name. The walk function provides you with the traversal, objects reduces the action to objects:

    walk(objects |= with_entries(
      if .key == "canStarUserSeeFreeRoomUpgrade" then .key = "freeroom_upgrade"
      elif .key == "bundleId" then .key = "bundle_id"
      # and so on...
      else . end
    ))