I am trying to make ONE session and keeps it alive for the whole iteration of payload.
The problem is if I make a session then iterate the payload, the session would also be iterated so I wouldn't get any benefit of using a session.
This is my code:
while i > 1 and i < 1570:
i += 1
payload = {
'number1': i,
}
r = requests.post(url, data=payload)
soup = BeautifulSoup(r.content, "lxml")
Getting your session and sending your request are two separated steps, so you can totally have one session for your script and send several requests with it:
with requests.Session() as s:
while i > 1 and i < 1570:
i += 1
payload = {
'number1': i,
}
r = s.post(url, data=payload)
soup = BeautifulSoup(r.content, "lxml")