so I've been having some issues trying to login to a site using httpwebrequests in vb.net. The site isn't anything amazing so it is definitely possible to log in. But what I am having issues with is the cookie container. I have been told by one person to not worry about cookies and not to include them, whereas someone else has told me just make a variable as a new cookiecontainer and it will store all of the cookies from the site without you having to put them in manually. But in order to log in to the site, the post data requires the csrftoken which is stored in the cookie. However, it can also be scraped from the source code. But I want to know how the cookies get from the site I'm trying to log in on into my cookie container before I have logged in so I can use the csrftoken from the cookie in the post data so I can log in? Here is my code: (note I scraped the csrftoken because I didn't know how to get it from the container)
Imports System.Net
Imports System.Text
Imports System.IO
Imports HtmlAgilityPack
Public Class Form1
Dim csrftoken As String
Dim logincookie As New CookieContainer
Dim uri As Uri = New Uri("site")
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim sourceString As String = New System.Net.WebClient().DownloadString("site")
Dim doc As New HtmlDocument
doc.LoadHtml(sourceString)
For Each node As HtmlAgilityPack.HtmlNode In doc.DocumentNode.SelectNodes("//*[@id=""login_inputs""]/form/input[1]")
Dim csrftokenmixed As String = node.OuterHtml
Dim cutat As String = "value='"
Dim x As Integer = InStr(csrftokenmixed, cutat)
Dim csrftoken1 As String = csrftokenmixed.Substring(x + cutat.Length - 1)
Dim cutat2 As String = "'"
Dim x2 As Integer = InStr(csrftoken1, cutat2)
csrftoken = csrftoken1.Substring(0, x2 - 1)
Next
'Note I know this is increadibly poorly coded but I was just testing it out and I just copied and pasted some code in order to get the value.
Dim username As String = TextBox1.Text
Dim password As String = TextBox2.Text
Dim postData As String = "next=&csrfmiddlewaretoken=" & csrftoken & "&username=" & username & "&password=" & password
Dim tempCookies As New CookieContainer
Dim encoding As New UTF8Encoding
Dim byteData As Byte() = encoding.GetBytes(postData)
Dim postReq As HttpWebRequest = DirectCast(WebRequest.Create("site"), HttpWebRequest)
postReq.Method = "POST"
postReq.Host = "site.com"
postReq.KeepAlive = True
postReq.ContentLength = byteData.Length
postReq.Headers.Add("Cache-Control", "max-age=0")
postReq.Headers.Add("Origin", "site")
postReq.Headers.Add("Upgrade-Insecure-Requests", "1")
postReq.ContentType = "application/x-www-form-urlencoded"
postReq.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
postReq.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
postReq.Referer = "site"
postReq.Headers.Add("Accept-Encoding", "gzip, deflate, br")
postReq.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6")
*tempCookies*.Add(uri, New Cookie("csrftoken", csrftoken))
postReq.CookieContainer = tempCookies
Dim postreqstream As Stream = postReq.GetRequestStream()
postreqstream.Write(byteData, 0, byteData.Length)
postreqstream.Close()
For Each tempCookie In *tempCookies*.GetCookies(uri)
Debug.Print(tempCookie.name & " = " & tempCookie.value)
Next
This is where the code gets stuck trying to get the response - Error 403 The remote server returned an error - forbidden.
Dim postresponse As HttpWebResponse
postresponse = DirectCast(postReq.GetResponse(), HttpWebResponse)
tempCookies.Add(postresponse.Cookies)
logincookie = tempCookies
Dim postreqreader As New StreamReader(postresponse.GetResponseStream())
Dim thepage As String = postreqreader.ReadToEnd
WebBrowser1.DocumentText = thepage
End Sub
End Class
So Yeah I forgot to mention that I can't actually get a response - I think its due to my cookies but I don't know? Any help is appreciated on where I should be going.
Also before anyone says make sure all your headers are the same - I have multiple times and the only think that I can't replicate are teh cookies.
Update: Just realised that I was using logincookie as the cookiecontainer instead of the tempCookies - so I basically replace all the logincookies with tempCookies below where in the second section of my code. But all the data comes out looking like this:
in short answer:
csrftoken can be get in response header.
so before doing a POST,just do a GET, then in response you will get and have csrftoken , then use it in POST parameters ...