I have a few convenience funcs and types in my utils
package I use in several projects. They help me getting objects, array of objects from external apis via urls or custom requests (for auth api parts). Here how they look:
var myClient = &http.Client{Timeout: 10 * time.Second}
func GetJsonAsMap(url string) (hashmap Map, err error) {
err = GetJson(url, &hashmap)
return
}
func GetJsonAsMapArray(url string) (mapArray []Map, err error) {
err = GetJson(url, &mapArray)
return
}
// Low-level func. Target must be a pointer.
func GetJson(url string, target any) error {
req, err := http.NewRequest("GET", url, nil)
// commented out error handling
resp, err := myClient.Do(req)
// commented out error handling
defer resp.Body.Close()
err = json.NewDecoder(resp.Body).Decode(target)
// commented out error handling
return nil
}
It worked great! Lower level func could also be used when I need to populate a well-defined struct. But most of the time I would just use map or maparray convenience funcs because I want to keep pristine copies of objects from api reply, where new fields could be added to objects by api and I don't want to miss them if I don't update associated struct.
This was good times, when everything just worked.
Later I needed to do a few requests simultaneously, as there is no need to send them one after another, this would drastically shorten script execution. And this is what I tried. I added two new multiple
versions of the GetJsonAsMap
and GetJsonAsMapArray
functions:
func GetJsonAsMapMultiple(urls []string) (hashmaps []Map, errs []error) {
l := len(urls)
hashmaps = make([]Map, l)
errs = make([]error, l)
ch := make(chan int, l)
for i, url := range urls {
go func() {
hashmaps[i], errs[i] = GetJsonAsMap(url)
ch <- i
}()
}
for range l {
<-ch
}
return
}
func GetJsonAsMapArrayMultiple(urls []string) (mapArrays [][]Map, errs []error) {
l := len(urls)
mapArrays = make([][]Map, l)
errs = make([]error, l)
ch := make(chan int, l)
for i, url := range urls {
go func() {
mapArrays[i], errs[i] = GetJsonAsMapArray(url)
ch <- i
}()
}
for range l {
<-ch
}
return
}
Here you immediately see a problem. These are two identical funcs that just differ in types and in underlying helper func they call. I am also limited in that I don't have lower level multiple version of the func (and can't use well-defined structs to populate in multiple fashion).
I think I might have just one low level multiple func returning some generic type, but I'm not sure how to go about that. Maybe there is a better approach. Right now these multiple funcs are plain ugly.
Use type parameters to eliminate code duplication:
// GetJson decodes the resource at url to T and returns the result.
func GetJson[T any](url string) (T, error) {
req, err := http.NewRequest("GET", url, nil)
// commented out error handling
resp, err := myClient.Do(req)
// commented out error handling
defer resp.Body.Close()
var target T
err = json.NewDecoder(resp.Body).Decode(target)
// commented out error handling
return target, err
}
// GetJsons decodes each resource at urls to a T and returns
// a slice of the results.
func GetJsons[T any](urls []string) ([]T, []error) {
errors := make([]error, len(urls))
targets := make([]T, len(urls))
var wg sync.WaitGroup
wg.Add(len(urls))
for i, url := range urls {
go func() {
defer wg.Done()
targets[i], errors[i] = GetJson[T](url)
}()
}
wg.Wait()
return targets, errors
}
Example use:
hashmaps, errors := GetJsons[Map](urls)