jsonbashxidel

xidel: is it possible to retrieve specific nested values from JSON object?


Consider this json, from this question

{
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "metadata": {
        "annotations": {
            "deployment.kubernetes.io/revision": "1"
        },
        "creationTimestamp": "2023-02-22T17:35:22Z",
        "generation": 1,
        "labels": {
            "app": "addresses",
            "group": "de.test",
            "provider": "jkube",
            "version": "2.3.0-3191"
        },
        "name": "addresses",
        "namespace": "linkerd",
        "resourceVersion": "910972",
        "uid": "aaee0eb0-5d66-44dd-b9b0-35e17bec4cf4"
    },
    "spec": {
        "progressDeadlineSeconds": 600,
        "replicas": 1,
        "revisionHistoryLimit": 2,
        "selector": {
            "matchLabels": {
                "app": "addresses",
                "group": "de.test",
                "provider": "jkube"
            }
        },
        "strategy": {
            "rollingUpdate": {
                "maxSurge": 1,
                "maxUnavailable": "50%"
            },
            "type": "RollingUpdate"
        },
        "template": {
            "metadata": {
                "annotations": {
                    "jkube.io/git-branch": "develop",
                    "jkube.io/git-commit": "77aaca38564dffff0ce10a8c70d4139b33f677b2",
                    "jkube.io/git-url": "http://bitbucket.org/testzentrale/test-monorepo",
                    "jkube.io/scm-tag": "HEAD",
                    "jkube.io/scm-url": "https://bitbucket.org/testzentrale/test-monorepo/test/addresses"
                },
                "creationTimestamp": null,
                "labels": {
                    "app": "addresses",
                    "group": "de.test",
                    "provider": "jkube",
                    "version": "2.3.0-3191"
                },
                "namespace": "test-testing"
            },
            "spec": {
                "containers": [
                    {
                        "env": [
                            {
                                "name": "SENTRY_ENVIRONMENT",
                                "valueFrom": {
                                    "fieldRef": {
                                        "apiVersion": "v1",
                                        "fieldPath": "metadata.namespace"
                                    }
                                }
                            },
                            {
                                "name": "SENTRY_SERVERNAME",
                                "valueFrom": {
                                    "fieldRef": {
                                        "apiVersion": "v1",
                                        "fieldPath": "metadata.name"
                                    }
                                }
                            },
                            {
                                "name": "SENTRY_TAGS",
                                "value": "service:addresses"
                            },
                            {
                                "name": "SENTRY_RELEASE",
                                "value": "2.3.0-3191"
                            },
                            {
                                "name": "JAVA_OPTIONS",
                                "value": "-Xms64m -Xmx384m"
                            },
                            {
                                "name": "KUBERNETES_NAMESPACE",
                                "valueFrom": {
                                    "fieldRef": {
                                        "apiVersion": "v1",
                                        "fieldPath": "metadata.namespace"
                                    }
                                }
                            },
                            {
                                "name": "HOSTNAME",
                                "valueFrom": {
                                    "fieldRef": {
                                        "apiVersion": "v1",
                                        "fieldPath": "metadata.name"
                                    }
                                }
                            }
                        ],
                        "envFrom": [
                            {
                                "configMapRef": {
                                    "name": "db-mssql-asd-connection"
                                }
                            },
                            {
                                "secretRef": {
                                    "name": "db-mssql-asd-connection"
                                }
                            },
                            {
                                "configMapRef": {
                                    "name": "db-postgre-asd-connection"
                                }
                            },
                            {
                                "secretRef": {
                                    "name": "db-postgre-asd-connection"
                                }
                            },
                            {
                                "configMapRef": {
                                    "name": "app-url"
                                }
                            },
                            {
                                "secretRef": {
                                    "name": "app-token"
                                }
                            }                                                      
                        ],
                        "image": "myrepo/test/addresses:testing-2.3.0-3191",
                        "imagePullPolicy": "IfNotPresent",
                        "livenessProbe": {
                            "failureThreshold": 3,
                            "httpGet": {
                                "path": "/",
                                "port": 8080,
                                "scheme": "HTTP"
                            },
                            "initialDelaySeconds": 300,
                            "periodSeconds": 10,
                            "successThreshold": 1,
                            "timeoutSeconds": 1
                        },
                        "name": "spring-boot",
                        "ports": [
                            {
                                "containerPort": 8080,
                                "name": "http",
                                "protocol": "TCP"
                            },
                            {
                                "containerPort": 9779,
                                "name": "prometheus",
                                "protocol": "TCP"
                            },
                            {
                                "containerPort": 8778,
                                "name": "jolokia",
                                "protocol": "TCP"
                            }
                        ],
                        "readinessProbe": {
                            "failureThreshold": 2,
                            "httpGet": {
                                "path": "/actuator/health",
                                "port": 8080,
                                "scheme": "HTTP"
                            },
                            "initialDelaySeconds": 20,
                            "periodSeconds": 10,
                            "successThreshold": 1,
                            "timeoutSeconds": 1
                        },
                        "resources": {
                            "limits": {
                                "memory": "1000Mi"
                            },
                            "requests": {
                                "cpu": "100m",
                                "memory": "500Mi"
                            }
                        },
                        "securityContext": {
                            "privileged": false
                        },
                        "terminationMessagePath": "/dev/termination-log",
                        "terminationMessagePolicy": "File",
                        "volumeMounts": [
                            {
                                "mountPath": "/deployments/config",
                                "name": "config"
                            }
                        ]
                    }
                ],
                "dnsPolicy": "ClusterFirst",
                "restartPolicy": "Always",
                "schedulerName": "default-scheduler",
                "securityContext": {},
                "terminationGracePeriodSeconds": 30,
                "volumes": [
                    {
                        "configMap": {
                            "defaultMode": 420,
                            "items": [
                                {
                                    "key": "application.properties",
                                    "path": "application.properties"
                                }
                            ],
                            "name": "addresses"
                        },
                        "name": "config"
                    }
                ]
            }
        }
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2023-02-22T17:35:23Z",
                "lastUpdateTime": "2023-02-22T17:35:23Z",
                "message": "Deployment does not have minimum availability.",
                "reason": "MinimumReplicasUnavailable",
                "status": "False",
                "type": "Available"
            },
            {
                "lastTransitionTime": "2023-02-22T17:45:24Z",
                "lastUpdateTime": "2023-02-22T17:45:24Z",
                "message": "ReplicaSet \"addresses-dd575cdc9\" has timed out progressing.",
                "reason": "ProgressDeadlineExceeded",
                "status": "False",
                "type": "Progressing"
            }
        ],
        "observedGeneration": 1,
        "replicas": 1,
        "unavailableReplicas": 1,
        "updatedReplicas": 1
    }
}

You can download it via this JSON gist

curl -s https://gist.githubusercontent.com/sputnick-dev/7da057b951c182785dfb837194388350/raw/eaf064c6b44c8304e111fa4e6ee3638d47a4ef48/object.json > object.json

Is there a way to fetch/extract all the name properties of the configMapRef ?

(Searched here tags [xidel] [json], only two matches).

Expected output:

db-mssql-asd-connection
db-postgre-asd-connection
app-url

A simple case where I get xidel to parse json, with shortened sample:

$ cat object.json 
{
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "metadata": {
        "annotations": {
            "deployment.kubernetes.io/revision": "1"
        }
    }
}
$ xidel -e '($json).metadata.annotations["deployment.kubernetes.io/revision"]' -s object.json 
{
  "deployment.kubernetes.io/revision": "1"
}

Solution

  • Is there a way to fetch/extract all the name properties of the configMapRef ?

    Sure there is:

    $ xidel -s --input-format=json "https://gist.githubusercontent.com/sputnick-dev/7da057b951c182785dfb837194388350/raw/eaf064c6b44c8304e111fa4e6ee3638d47a4ef48/object.json" \
      -e '($json).spec.template.spec.containers().envFrom().configMapRef.name'     # dot notation
      -e '$json/spec/template/spec/(containers)()/(envFrom)()/configMapRef/name'   # XPath-like notation
      -e '$json?spec?template?spec?containers?*?envFrom?*?configMapRef?name'       # XPath 3.1 syntax
    

    (The Github gist-url is JSON, but $headers returns Content-Type: text/plain; charset=utf-8, instead of Content-Type: application/json, so --input-format=json to force it.)

    And you can shorten the XPath-like notation to:

    -e '$json/(.//envFrom)()/configMapRef/name'
    -e '($json//envFrom)()/configMapRef/name'