JSON, key-value-pairs in groups to "flat" key-value-pair

Each key of a JSON should be combined with _ (or any valid JSON symbol). To have a simple key-value list.

I have the following structure. A few JSON groups (no arrays), inside the groups there are key-value pairs. I need to flat them to a single key-value-list. I tried jq but there is only something "nested" / "unnested". I do not find something about flatten or compaine the keys.

So it should be "key_subkey_subsubkey": "value"

{
    "welcome": {
        "title" : "Hello World"
    },
    "block1": {
        "header": "My Header",
        "body": "My BODY of block 1",
        "footer": "My Footer"
    },
    "multi": {
        "level-01-A": {
            "head": "Head Section",
            "foot": "Foot Section"
            "level-02-A": {
                "head": "Head Section Level 2 A",
                "fead": "Foot Section Level 2 A"
            },
            "level-02-B": {
                "head": "Head Section Level 2 B",
                "fead": "Foot Section Level 2 B"
            },
        },
        "level-01-B": {
            "head": "Head Section",
            "foot": "Foot Section"
        }
        "no-level" : "Foo Bar",     
    }
}

and I want to have

{
    "welcome_title" : "Hello World",

    "block1_header": "My Header",
    "block1_body": "My BODY of block 1",
    "block1_footer": "My Footer",

    "multi_level-01-A_head": "Head Section",
    "multi_level-01-A_foot": "Foot Section",
        
    "multi_level-01-A_level-02-A_head": "Head Section Level 1 A",
    "multi_level-01-A_level-02-A_fead": "Foot Section Level 1 A",

    "multi_level-01-A_level-02-B_head": "Head Section Level 1 B",
    "multi_level-01-A_level-02-B_fead": "Foot Section Level 1 B",

    "multi_level-01-B_head": "Head Section",
    "multi_level-01-B_foot": "Foot Section",

    "multi_no-level" : "Foo Bar"
}

Any idea, what tool I can use?

Solution

[ paths(scalars) as $p | { "key": $p | join("_"), "value": getpath($p) } ] | from_entries

Will generate

{
  "welcome_title": "Hello World",
  "block1_header": "My Header",
  "block1_body": "My BODY of block 1",
  "block1_footer": "My Footer",
  "multi_level-01-A_head": "Head Section",
  "multi_level-01-A_foot": "Foot Section",
  "multi_level-01-A_level-02-A_head": "Head Section Level 2 A",
  "multi_level-01-A_level-02-A_fead": "Foot Section Level 2 A",
  "multi_level-01-A_level-02-B_head": "Head Section Level 2 B",
  "multi_level-01-A_level-02-B_fead": "Foot Section Level 2 B",
  "multi_level-01-B_head": "Head Section",
  "multi_level-01-B_foot": "Foot Section",
  "multi_no-level": "Foo Bar"
}

As you can test in this online demo

paths ^[docs]

paths outputs the paths to all the elements in its input (except it does not output the empty list, representing . itself).

getpath ^[docs]

The builtin function getpath outputs the values in . found at each path in PATHS.

from_entries ^[docs]

These functions convert between an object and an array of key-value pairs. If to_entries is passed an object, then for each k: v entry in the input, the output array includes {"key": k, "value": v}.

So the steps I took are:

Get every available (nested) path and hold it in a variable
```
paths(scalars) as $p
```
Create an object holding the key and value. We can retreive the value by using getpath
```
{ "key": $p | join("_"), "value": getpath($p) }
```
Use from_entries to convert to a single object
```
| from_entries
```