After reading some examples, I wanted to implement a crawler for helpshift with login such as:
https://target.helpshift.com/login/?next=%2Fadmin%2Fissues%2F
import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
public class JsouptTest {
public static void main(String[] args) throws Exception {
int x = 1;
Connection.Response loginForm = Jsoup.connect("https://target.helpshift.com/login/?next=%2Fadmin%2Fissues%2F" + x + "%2F")
.method(Connection.Method.GET)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0")
.execute();
Document document = Jsoup.connect("https://target.helpshift.com/login/")
.data("cookieexists", "false")
.data("username", "email@example.com")
.data("password", "123456")
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0")
.cookies(loginForm.cookies())
.post();
System.out.println(document);
}
}
However, I am getting this error:
Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=https://target.helpshift.com/login/ at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:537) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205) at org.jsoup.helper.HttpConnection.post(HttpConnection.java:200) at edu.utfpr.helpcrawler.JsouptTest.main(JsouptTest.java:32)
If you check the request headers you will see the it sends the cookies as you've done, but it includes a part of the cookie in the form data too. Add this to your second request
.data("_csrf_token", loginForm.cookie("_csrf_token"))