I have 2 methods in 2 different packages, where func B() takes a url reads the web page and returns *html.Tokenizer. But the problem is, it is working fine Only when I comment the defer r.Body.Close(), If I enable it this doc returned from func B is empty.
And it also works if both the functions are merged in single function. but I need them in 2 different package.
Any suggestion or Idea that What am I missing here ? shoudn't the res.Body be closed ?
func (s ParserService) A(u string) (*domain.Result, error) {
doc, err := s.B("https://www.google.com/")
if err != nil {
fmt.Println(err.Error())
}
for tokenType := doc.Next(); tokenType != html.ErrorToken; {
token := doc.Token()
fmt.Println(token)
tokenType = doc.Next()
}
}
func (c Downloader) B(url string) (*html.Tokenizer, error) {
r, err := c.httpClient.Get(url)
if err != nil {
return nil, err
}
// defer r.Body.Close()
doc := html.NewTokenizer(r.Body)
return doc, nil
}
The html.Tokenier
‘s Next
method reads directly from the reader. Don’t close the body until you’ve finished processing it through the tokenizer. In your example, you should perform the HTTP request and tokenize the body in the same function, then you can uncomment your deferred close.
html.Tokenizer
accepts an io.Reader
from which the tokenizer will read until it receives an io.EOF
error. This "error" indicates that there is nothing left to be read and the tokenizer source is completed.
http.Request.Body
is an io.ReadCloser
which is a combination of an io.Reader
and an io.Closer
. What happens after a call to Close
is implementation specific, however for the http.Request.Body, no more bytes can be read from the reader after close is called.
Your problem is ultimately caused by prematurely closing the http.Request.Body
(io.ReadCloser
).