gogenericsslicegob

Go: Implementing a ManyDecode for a "set" of individual results


I have implemented a very simple Decode method (using gob.Decoder for now) - this works well for single responses - it would even work well for slices, but I need to implement a DecodeMany method where it is able to decode a set of individual responses (not a slice).

Working Decode method:

var v MyType
_ = Decode(&v)
...

func Decode(v interface{}) error {
   buf, _ := DoSomething() // func DoSomething() ([]byte, error)
   // error handling omitted for brevity
   return gob.NewDecoder(bytes.NewReader(buf)).Decode(v)
}

What I'm trying to do for a DecodeMany method is to deal with a response that isn't necessarily a slice:

var vv []MyType
_ = DecodeMany(&vv)
...

func DecodeMany(vv []interface{}) error {
   for _, g := range DoSomething() { // func DoSomething() []struct{Buf []bytes}
      
      // Use g.Buf as an individual "interface{}"
      // want something like:
      var v interface{} /* Somehow create instance of single vv type? */
      _ = gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(v)
      vv = append(vv, v)
   }
   return
}

Besides not compiling the above also has the error of:

cannot use &vv (value of type *[]MyType) as type []interface{} in argument to DecodeMany


Solution

  • If you want to modify the passed slice, it must be a pointer, else you must return a new slice. Also if the function is declared to have a param of type []interface{}, you can only pass a value of type []interface{} and no other slice types... Unless you use generics...

    This is a perfect example to start using generics introduced in Go 1.18.

    Change DecodeMany() to be generic, having a T type parameter being the slice element type:

    When taking a pointer

    func DecodeMany[T any](vv *[]T) error {
        for _, g := range DoSomething() {
            var v T
            if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
                return err
            }
            *vv = append(*vv, v)
        }
        return nil
    }
    

    Here's a simple app to test it:

    type MyType struct {
        S int64
    }
    
    func main() {
        var vv []MyType
        if err := DecodeMany(&vv); err != nil {
            panic(err)
        }
        fmt.Println(vv)
    }
    
    func DoSomething() (result []struct{ Buf []byte }) {
        for i := 3; i < 6; i++ {
            buf := &bytes.Buffer{}
            v := MyType{S: int64(i)}
            if err := gob.NewEncoder(buf).Encode(v); err != nil {
                panic(err)
            }
            result = append(result, struct{ Buf []byte }{buf.Bytes()})
        }
        return
    }
    

    This outputs (try it on the Go Playground):

    [{3} {4} {5}]
    

    When returning a slice

    If you choose to return the slice, you don't have to pass anything, but you need to assign the result:

    func DecodeMany[T any]() ([]T, error) {
        var result []T
        for _, g := range DoSomething() {
            var v T
            if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
                return result, err
            }
            result = append(result, v)
        }
        return result, nil
    }
    

    Using it:

    vv, err := DecodeMany[MyType]()
    if err != nil {
        panic(err)
    }
    fmt.Println(vv)
    

    Try this one on the Go Playground.