[SOLVED] go-colly: slice as data into POST-request

go-colly: slice as data into POST-request

I want to use the calculations done by this website: http://www.isewase.de/dwz/.

Therefore I checked the POST-request when entering the data on the website itself, e.g.:

dwz_name: 1000-100
geb: 1990
punkte: 1
dwz[]: 1300
dwz[]: 1400
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
dwz[]: 
Los: DWZ berechnen!
name: 
kennwort:

or (view as source):

dwz_name=1000-100&geb=1990&punkte=1&dwz%5B%5D=1300&dwz%5B%5D=1400&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&dwz%5B%5D=&Los=DWZ+berechnen%21&name=&kennwort=

So I tried replicating it with this small program:

// test.go
package main

import (
    "fmt"
    "strings"

    "github.com/gocolly/colly"
)

func main() {
    dwz := [18]string{
        "1300",
        "1400",
    }

    dwzStr := strings.Join(dwz[:], "&dwz[]=") // (a)
    // dwzStr := strings.Join(dwz[:], "&dwz%5B%5D=") // (b)

    c := colly.NewCollector()
    c.OnHTML("body > form > fieldset:nth-child(4) > dl > dd:nth-child(8)", func(h *colly.HTMLElement) {
        fmt.Println(h.Text)
    })

    c.Post("http://www.isewase.de/dwz/", map[string]string{
        "dwz_name": "1000-100",
        "geb":      "1900",
        "punkte":   "1",
        "dwz[]":    dwzStr,
        "Los":      "DWZ berechnen!",
        "name":     "",
        "kennwort": "",
    })
}

with the result (for both (a) and (b))

> go run test.go
1040 (+40)

which is exactly the result you receive from the website when just entering the first oppenent with 1300.

But since the colly.Post wants a map[string]string as second argument, I don't know how to pass the array/slice another way into the POST-request.

Solution

You should be able to use c.Request so you can manually set the request body:

form := url.Values{}
form.Add("dwz_name", "1000-100")
form.Add("geb", "1900")
form.Add("punkte", "1")
form.Add("Los", "DWZ berechnen!")
form.Add("name", "")
form.Add("kennwort", "")
form.Add("dwz[]", "1300")
form.Add("dwz[]", "1400")
header := http.Header{}
header.Add("Content-Type", "application/x-www-form-urlencoded")

err := c.Request(
    http.MethodPost, 
    "http://www.isewase.de/dwz/", 
    strings.NewReader(form.Encode()), 
    nil,
    header,
)