I have an API that often returns arrays as an object that contains an array. Take the example below:
{
"items": {
"number": 3,
"item": [
{ ... } // Not relevant
]
}
}
The API does this in dozens of places with a different name each time. It is guaranteed that when this occurs there are only two keys: one of them being number
and the other being the array.
This makes the resulting structs rather unpleasant to work with, as you constantly have to navigate through levels of unnecessary fields.
I essentially want my Go interface to pretend it had this format instead:
{
"items": [
{ ... } // Not relevant
]
}
One option is to write a custom UnmarshalJSON
function for every single occurrence, but this seems cumbersome, especially considering this appears in nearly every struct. The solution I had in mind is a generic type that can handle it on its own.
My current attempt is below:
// NestedArray tries to pull an unnecessarily nested array upwards
type NestedArray[T any] []T
func (n *NestedArray[T]) UnmarshalJSON(bytes []byte) error {
// First unmarshal into a map
target := make(map[string]interface{})
err := json.Unmarshal(bytes, &target)
if err != nil {
return err
}
// Then find the nested array (key is unknown, so go off of the type instead)
var sliceVal interface{}
for k, v := range target {
if k == "number" {
continue
}
rt := reflect.TypeOf(v)
if rt.Kind() == reflect.Slice {
sliceVal = v
break
}
}
// Missing or empty, doesn't matter - set the result to nil
if sliceVal == nil {
*n = nil
return nil
}
// Turn back into JSON and parse into correct target
sliceJSON, err := json.Marshal(sliceVal)
if err != nil {
return err
}
err = json.Unmarshal(sliceJSON, n) // Error occurs here
if err != nil {
return err
}
return nil
}
Using it as follows:
type Item struct {
// Not relevant
}
type Root struct {
// Use generic type to parse a JSON object into its nested array
Items NestedArray[Item] `json:"items,omitempty"`
}
Results in the following error:
json: cannot unmarshal array into Go struct field Root.items of type map[string]interface{}
The biggest part of UnmarshalJSON
code seems correct, as my debugger shows me that sliceVal
is what I'd expect it to be. It errors when unmarshalling back into the NestedArray[T]
type.
What could be the solution to this problem? Is there a better way to go about it than what I'm currently doing? This seemed the cleanest to me but I'm open to suggestions.
The method NestedArray[T].UnmarshalJSON calls itself recursively. The inner call throws an error because it's expecting a JSON object in bytes
, but it received a JSON array. Fix by unmarshalling to a []T
instead of a NestedArray[T]
.
Unrelated to the error, the method NestedArray[T].UnmarshalJSON does some unnecessary encoding and decoding. Fix by using json.RawMessage.
Here's the code with both fixes:
func (n *NestedArray[T]) UnmarshalJSON(bytes []byte) error {
// First unmarshal into a map
var target map[string]json.RawMessage
err := json.Unmarshal(bytes, &target)
if err != nil {
return err
}
// Then find the nested array (key is unknown, so go off of the type instead)
var array json.RawMessage
for k, v := range target {
if k == "number" {
continue
}
if len(v) > 0 && v[0] == '[' {
array = v
break
}
}
// Missing or empty, doesn't matter - set the result to nil
if array == nil {
*n = nil
return nil
}
// Avoid recursive call to this method by unmarshalling to a []T.
var v []T
err = json.Unmarshal(array, &v)
*n = v
return err
}