sqlmysqlarraysjsonjson-table

Extract JSON array values with generic keys to JSON_TABLE in uniform columns


I have JSON that is being brought into MariaDB/MYSQL database via Airbyte from a API query. I am using JSON_TABLE to break the JSON into columns - which is mostly working great, but I just noticed that there is an array of JSON elements that changes from time to time. Rather than using unique keypairs, they are sending generic keypairs, for example:

{
  "id": 123,
  "position": "North",
  "attributes": [{
    "name": "First Name",
    "value": "Santa"
  }, {
    "name": "Last Name",
    "value": "Claus"
  }]
}

The elements can appear in any order so I can't target them with my JSON_TABLE query:

SELECT t.id, t.position, t.firstName, t.lastName 
FROM
    (
    mytable
    JOIN JSON_TABLE(mytable, '$'
    COLUMNS (
            id int(10) PATH '$.id',
            position varchar(20) PATH '$.position',
            firstName varchar(20) PATH '$.attributes[0].value',
            lastName varchar(20) PATH '$.attributes[1].value'
            )
          ) t
     ) 

How can I handle the attributes returning in a dynamic order?

Also, the amount of elements may change as well, so whatever i use to target the elements must fail gracefully if the elements don't exist.

My plan was to utilize JSON_SEARCH, so search for the object name = "First Name" then return the value to the element. I cannot figure out how to do this within a JSON_TABLE Query. I've tried:

First Name varchar(20) PATH REPLACE(JSON_SEARCH('$.attributes', 'all', 'First Name'), '.name', '.value'))

Which in a standard query will return the element path. However i need to dynamically build a JSON_TABLE for this to work properly. I get a SQL error near REPLACE(JSON_SEARCH which I don't have a working example to compare against.


Solution

  • The idea is to use json_table to convert the $.attributes array into rows. You could use the nested path clause, a sub query inside select, etc. Rest is straight forward:

    select
        t.pk,
        jt.id,
        jt.position,
        min(case when jt.name = 'first name' then jt.value end) as fname,
        min(case when jt.name = 'last name' then jt.value end) as lname
    from t
    cross join json_table(
        t.js,
        '$' columns (
            id int path '$.id',
            position varchar(20) path '$.position',
            nested path '$.attributes[*]' columns (
                name varchar(20) path '$.name',
                value varchar(20) path '$.value'
            )
        )
    ) as jt
    group by t.pk, jt.id, jt.position
    

    DB<>Fiddle