I'm trying to get specific table to loop through its content using colly but table its not being recognized, here's what I have so far.
package main
import (
"fmt"
"github.com/gocolly/colly"
)
func main() {
c := colly.NewCollector(
colly.AllowedDomains("wikipedia.org", "en.wikipedia.org"),
)
links := make([]string, 0)
c.OnHTML("div.mw-parser-output", func(e *colly.HTMLElement) {
e.ForEach("table.wikitable.sortable.jquery-tablesorter > tbody > tr", func(_ int, elem *colly.HTMLElement) {
fmt.Println(elem.ChildAttr("a[href]", "href"))
links = append(links, elem.ChildAttr("a[href]", "href"))
})
})
c.OnRequest(func(r *colly.Request) {
fmt.Println("Visiting", r.URL.String())
})
c.Visit("https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population")
fmt.Println("Found urls for", len(links), "countries.")
}
I need to loop thought all of the tr elements in the the table.
Turns out name of the class is actually wikitable.sortable
even though in chrome console is shown as wikitable sortable jquery-tablesorter
. I dont know why the names are different like this but it solved the problem for me.