gounmarshalling

Generic type definition to unmarshal struct as slice


I have an API that often returns arrays as an object that contains an array. Take the example below:

{
  "items": {
    "number": 3,
    "item": [
      { ... } // Not relevant
    ]
  }
}

The API does this in dozens of places with a different name each time. It is guaranteed that when this occurs there are only two keys: one of them being number and the other being the array.

This makes the resulting structs rather unpleasant to work with, as you constantly have to navigate through levels of unnecessary fields.

I essentially want my Go interface to pretend it had this format instead:

{
  "items": [
    { ... } // Not relevant
  ]
}

One option is to write a custom UnmarshalJSON function for every single occurrence, but this seems cumbersome, especially considering this appears in nearly every struct. The solution I had in mind is a generic type that can handle it on its own.

My current attempt is below:

// NestedArray tries to pull an unnecessarily nested array upwards
type NestedArray[T any] []T

func (n *NestedArray[T]) UnmarshalJSON(bytes []byte) error {
    // First unmarshal into a map
    target := make(map[string]interface{})

    err := json.Unmarshal(bytes, &target)
    if err != nil {
        return err
    }

    // Then find the nested array (key is unknown, so go off of the type instead)
    var sliceVal interface{}
    for k, v := range target {
        if k == "number" {
            continue
        }

        rt := reflect.TypeOf(v)
        if rt.Kind() == reflect.Slice {
            sliceVal = v
            break
        }
    }

    // Missing or empty, doesn't matter - set the result to nil
    if sliceVal == nil {
        *n = nil
        return nil
    }

    // Turn back into JSON and parse into correct target
    sliceJSON, err := json.Marshal(sliceVal)
    if err != nil {
        return err
    }

    err = json.Unmarshal(sliceJSON, n)  // Error occurs here
    if err != nil {
        return err
    }

    return nil
}

Using it as follows:

type Item struct {
  // Not relevant
}

type Root struct {
    // Use generic type to parse a JSON object into its nested array
    Items NestedArray[Item] `json:"items,omitempty"`
}

Results in the following error:

json: cannot unmarshal array into Go struct field Root.items of type map[string]interface{}

The biggest part of UnmarshalJSON code seems correct, as my debugger shows me that sliceVal is what I'd expect it to be. It errors when unmarshalling back into the NestedArray[T] type.

What could be the solution to this problem? Is there a better way to go about it than what I'm currently doing? This seemed the cleanest to me but I'm open to suggestions.


Solution

  • The method NestedArray[T].UnmarshalJSON calls itself recursively. The inner call throws an error because it's expecting a JSON object in bytes, but it received a JSON array. Fix by unmarshalling to a []T instead of a NestedArray[T].

    Unrelated to the error, the method NestedArray[T].UnmarshalJSON does some unnecessary encoding and decoding. Fix by using json.RawMessage.

    Here's the code with both fixes:

    func (n *NestedArray[T]) UnmarshalJSON(bytes []byte) error {
        // First unmarshal into a map
        var target map[string]json.RawMessage
        err := json.Unmarshal(bytes, &target)
        if err != nil {
            return err
        }
    
        // Then find the nested array (key is unknown, so go off of the type instead)
        var array json.RawMessage
        for k, v := range target {
            if k == "number" {
                continue
            }
            if len(v) > 0 && v[0] == '[' {
                array = v
                break
            }
        }
    
        // Missing or empty, doesn't matter - set the result to nil
        if array == nil {
            *n = nil
            return nil
        }
    
        // Avoid recursive call to this method by unmarshalling to a []T.
        var v []T
        err = json.Unmarshal(array, &v)
        *n = v
        return err
    }
    

    Run the code on the playground!.