Go 1.18.1
pprof report
3549.93kB 49.73% 49.73% 3549.93kB 49.73% src/lag_monitor.PublishLagMetricToDataDog
514kB 7.20% 56.93% 514kB 7.20% bufio.NewWriterSize
512.88kB 7.18% 64.11% 512.88kB 7.18% encoding/pem.Decode
512.69kB 7.18% 71.30% 1536.98kB 21.53% crypto/x509.parseCertificate
512.50kB 7.18% 78.48% 512.50kB 7.18% crypto/x509.(*CertPool).AddCert
This piece of code appears to not release memory and based on pprof, the beloew function is the one consuming most memory.Memory graph
func caller() {
events := make([]string, 0)
//....
PublishLagMetricToDataDog(ctx, strings.Join(events, ","))
}
func PublishLagMetricToDataDog(ctx context.Context, events string) error {
msg := `{
"series": [%v]
}`
b := []byte(msg)
resp, err := http.Post("https://api.datadoghq.com/api/v1/series?api_key="+env.GetDataDogKey(), "application/json", bytes.NewBuffer(b))
if err != nil {
logger.Error(ctx, "Error submitting event to datadog, err = ", err)
return err
}
logger.Info(ctx, resp)
return nil
}
Above function is called in a loop. Since there are no global variables, and no reference to the byte slice from PublishLagMetricToDataDog, I am not able to pinpoint the memory leak. I read about Reset() and Truncate(), but this does not release the underlying memory.
You must close the response body for every http response you receive. Not doing so will potentially lead to resource leaks, such as the one you've observed.
Solution:
resp, err := http.Post("https://api.datadoghq.com/api/v1/series?api_key="+env.GetDataDogKey(), "application/json", bytes.NewBuffer(b))
if err != nil {
logger.Error(ctx, "Error submitting event to datadog, err = ", err)
return err
}
logger.Info(ctx, resp)
_ = resp.Body.Close() // <--- Add this
return nil
}