httpgoreferrer

How do you specify the http referrer in golang as client?


Using the standard http.Client, how do you build a web request that specifies a referrer in the http request header?

Below you can see it is possible to set headers, but how do you specify the referer? Is it just by setting a Referer header?

req, err := http.NewRequest("GET", url, nil)
if err != nil {
    return "", err
}
req.Header.Set("Accept", "text/html,application/xhtml+xml")
req.Header.Set("User-Agent", "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1)")

response, err1 := client.Get(url)
if err1 != nil {
    return "", err1
}

Solution

  • Yes, as you can see from the sources from Go itself, in src/net/http/client.go

    // Add the Referer header from the most recent
    // request URL to the new one, if it's not https->http:
    if ref := refererForURL(reqs[len(reqs)-1].URL, req.URL); ref != "" {
        req.Header.Set("Referer", ref)
    }
    

    Check your scheme though, as in the same sources:

    // refererForURL returns a referer without any authentication info or
    // an empty string if lastReq scheme is https and newReq scheme is http.
    func refererForURL(lastReq, newReq *url.URL) string {
        // https://tools.ietf.org/html/rfc7231#section-5.5.2
        //   "Clients SHOULD NOT include a Referer header field in a
        //    (non-secure) HTTP request if the referring page was
        //    transferred with a secure protocol."
        if lastReq.Scheme == "https" && newReq.Scheme == "http" {
            return ""
    }