goprofilingpprof

profiling http handler in go lang

I am trying to profile my http handler written in go. Which on every http request download an image from S3, resize it/crop it and write it in response.

I have followed this link and tried to profile my code as mentioned using easy method as well as hard method. Now, when i use the following line as mentioned in the code. 

defer profile.Start(profile.CPUProfile).Stop()

It doesn't write anything in the /tmp/profie[some number]/cpu.pprof file

func main() {                                    

       defer profile.Start(profile.CPUProfile).Stop()   

      if err := http.ListenAndServe(":8081", http.HandlerFunc(serveHTTP)); err != nil {
       logFatal("Error when starting or running http server: %v", err)
    }       
}

func serveHTTP(w http.ResponseWriter, r *http.Request) {
        keyName := r.URL.Path[1:]
        s3Client := s3.New(session.New(), &aws.Config{Region: aws.String(region)})
        params := &s3.GetObjectInput{
        Bucket: aws.String(bucketName),
        Key: aws.String(keyName),
        }

    mw := imagick.NewMagickWand()
    defer mw.Destroy()
    ...
}

Moreover, when i used the defer profile.Start(profile.CPUProfile).Stop() line inside the serveHTTP like :

func serveHTTP(w http.ResponseWriter, r *http.Request) {

    defer profile.Start(profile.CPUProfile).Stop()
    ......
}

It creates multiple files in the /tmp/profile[some number] folder. So, first question is why it is not writing in the file and secondly shouldn't it be places inside the serveHTTP method because server will get started only once. Hence main() will be called once wheres serveHTTP wil be called on every request.

Part 1 

 .               124:   s3Client := s3.New(session.New(), &aws.Config{Region: aws.String(region)})
 .          .    125:        params := &s3.GetObjectInput{
 .          .    126:       Bucket: aws.String(masterBucketName),
 .          .    127:       Key: aws.String(keyName),
 .    32.01kB    128:       }
 .          .    129:
 .          .    130:   mw := imagick.NewMagickWand()
 .          .    131:   defer mw.Destroy()
 .          .    132:   
 .          .    133:   out, err := s3Client.GetObject(params)          
 .          .    134:
 .          .    135:   if strings.EqualFold(keyName[strings.LastIndex(keyName,".")+1:len(keyName)], "gif") {
 .          .    136:
 .    40.11kB    137:       blobGiff, err := ioutil.ReadAll(out.Body)
 .          .    138:       w.Header().Set("Content-Type", "image/gif")
 .          .    139:       w.Header().Set("Cache-Control", "max-age: 604800, public")  
 .          .    140:       w.Header().Set("Last-Modified", time.Now().Format(http.TimeFormat))
 .          .    141:       w.Header().Set("Expires", time.Now().AddDate(1, 0, 0).Format(http.TimeFormat))  
 .          .    142:

Part 2 :

                                   else {
         .          .    167:       img, err := ioutil.ReadAll(out.Body)
         .          .    168:       if err != nil {
         .          .    169:          
         .          .    170:          w.WriteHeader(http.StatusNotFound) 
         .     1.56MB    171:          return      
         .          .    172:       }

Also, in the above two parts line 128, 137 and 171 has memory leaks, right? Also, I don't find any option to close/destroy the s3Client and blobGiff (byte []).

Solution

  • First of all use import "net/http/pprof" NOT import _ "net/http/pprof. later one didn't recognize the pprof in the below routes.

    I was using the default serveMux/multiplexer. But then I created my own as people suggested it has performance implication.

    myMux := http.NewServeMux()

    Then added the route for the request

    myMux.HandleFunc("/", serveHTTP)

    Morever, I also added the routes for to make the http://localhost:8081/debug/pprof/ work

            myMux.HandleFunc("/debug/pprof/", pprof.Index)
        myMux.HandleFunc("/debug/pprof/{action}", pprof.Index)
        myMux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)

    So, final code would be :

    import "net/http/pprof

    func main() {                                    
        
        myMux := http.NewServeMux()
        myMux.HandleFunc("/", serveHTTP)

        myMux.HandleFunc("/debug/pprof/", pprof.Index)
        myMux.HandleFunc("/debug/pprof/{action}", pprof.Index)
        myMux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)   

    if err := http.ListenAndServe(":8081", myMux); err != nil {
        logFatal("Error when starting or running http server: %v", err)
    }       

}