goextractmetagoquery

Extract meta description field with goquery


I'm using the goquery package to extract pieces of information from a webpage. Please see my code below. The outcome when after running the function is:

Description field: text/html; charset=iso-8859-15
Description field: width=device-width
Description field: THIS IS THE TEXT I WANT TO EXTRACT

I'm almost there, however I only want to get the meta field where the name == 'description'. Unfortunately I can't figure out how to add this extra condition to my code.

func ExampleScrapeDescription() {
    htmlCode :=
        `<!doctype html>
<html lang="NL">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=iso-8859-15">
        <meta name="viewport" content="width=device-width">
        <meta name="description" content="THIS IS THE TEXT I WANT TO EXTRACT">
        <title>page title</title>
    </head>
    <body class="fixedHeader">
        page body
    </body>
</html>`

    doc, err := goquery.NewDocumentFromReader(strings.NewReader((htmlCode)))
    if err != nil {
        log.Fatal(err)
    }

    doc.Find("meta").Each(func(i int, s *goquery.Selection) {
        description, _ := s.Attr("content")
        fmt.Printf("Description field: %s\n", description)
    })
}

Solution

  • Just examine the value of the name attribute whether it matches "description":

    doc.Find("meta").Each(func(i int, s *goquery.Selection) {
        if name, _ := s.Attr("name"); name == "description" {
            description, _ := s.Attr("content")
            fmt.Printf("Description field: %s\n", description)
        }
    })
    

    You may want to compare the value of the name attribute in a case insensitive manner, for that you can use strings.EqualFold():

    if name, _ := s.Attr("name"); strings.EqualFold(name, "description") {
        // proceed to extract and use the content of description
    }