gogoroutine

Functions look very similar. How to avoid code duplication?


I have a few convenience funcs and types in my utils package I use in several projects. They help me getting objects, array of objects from external apis via urls or custom requests (for auth api parts). Here how they look:

var myClient = &http.Client{Timeout: 10 * time.Second}

func GetJsonAsMap(url string) (hashmap Map, err error) {
    err = GetJson(url, &hashmap)
    return
}

func GetJsonAsMapArray(url string) (mapArray []Map, err error) {
    err = GetJson(url, &mapArray)
    return
}

// Low-level func. Target must be a pointer.
func GetJson(url string, target any) error {
    req, err := http.NewRequest("GET", url, nil)
    // commented out error handling

    resp, err := myClient.Do(req)
    // commented out error handling

    defer resp.Body.Close()

    err = json.NewDecoder(resp.Body).Decode(target)
    // commented out error handling

    return nil
}

It worked great! Lower level func could also be used when I need to populate a well-defined struct. But most of the time I would just use map or maparray convenience funcs because I want to keep pristine copies of objects from api reply, where new fields could be added to objects by api and I don't want to miss them if I don't update associated struct.

This was good times, when everything just worked.

Later I needed to do a few requests simultaneously, as there is no need to send them one after another, this would drastically shorten script execution. And this is what I tried. I added two new multiple versions of the GetJsonAsMap and GetJsonAsMapArray functions:

func GetJsonAsMapMultiple(urls []string) (hashmaps []Map, errs []error) {
    l := len(urls)
    hashmaps = make([]Map, l)
    errs = make([]error, l)
    ch := make(chan int, l)

    for i, url := range urls {
        go func() {
            hashmaps[i], errs[i] = GetJsonAsMap(url)
            ch <- i
        }()
    }

    for range l {
        <-ch
    }

    return
}

func GetJsonAsMapArrayMultiple(urls []string) (mapArrays [][]Map, errs []error) {
    l := len(urls)
    mapArrays = make([][]Map, l)
    errs = make([]error, l)
    ch := make(chan int, l)

    for i, url := range urls {
        go func() {
            mapArrays[i], errs[i] = GetJsonAsMapArray(url)
            ch <- i
        }()
    }

    for range l {
        <-ch
    }

    return
}

Here you immediately see a problem. These are two identical funcs that just differ in types and in underlying helper func they call. I am also limited in that I don't have lower level multiple version of the func (and can't use well-defined structs to populate in multiple fashion).

I think I might have just one low level multiple func returning some generic type, but I'm not sure how to go about that. Maybe there is a better approach. Right now these multiple funcs are plain ugly.


Solution

  • Use type parameters to eliminate code duplication:

    // GetJson decodes the resource at url to T and returns the result.
    func GetJson[T any](url string) (T, error) {
        req, err := http.NewRequest("GET", url, nil)
        // commented out error handling
    
        resp, err := myClient.Do(req)
        // commented out error handling
    
        defer resp.Body.Close()
    
        var target T
        err = json.NewDecoder(resp.Body).Decode(target)
        // commented out error handling
    
        return target, err
    }
    
    // GetJsons decodes each resource at urls to a T and returns
    // a slice of the results.
    func GetJsons[T any](urls []string) ([]T, []error) {
        errors := make([]error, len(urls))
        targets := make([]T, len(urls))
        var wg sync.WaitGroup
        wg.Add(len(urls))
        for i, url := range urls {
            go func() {
                defer wg.Done()
                targets[i], errors[i] = GetJson[T](url)
            }()
        }
        wg.Wait()
        return targets, errors
    }
    

    Example use:

    hashmaps, errors := GetJsons[Map](urls)