I've had a website scraper running for an extended time without any issues. Recently, I believe that the website made a change and I'm now having session persistence issues. Here is the code to create my browser:
def create_browser():
# Browser
br = mechanize.Browser()
# Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
# Browser options
br.set_handle_equiv(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
return br
and here's the code to log in:
def login_stubhub(br):
br.open("https://myaccount.stubhub.com/login/Signin?")
br.select_form(name="signinForm_0")
br["loginEmail"] = 'username'
br["loginPassword"] = 'password'
br.method = "POST"
response = br.submit()
When I try and open a page that requires me to log in:
br.open(https://sell.stubhub.com/sellapi/event/4459340/section/null/seatmapdata)
I get the following response:
<data>
<errors>
<FormErrors>
<FormField>User Auth Check</FormField>
<ErrorMessage>
Either is not active or the session might have expired. Please login again.
</ErrorMessage>
</FormErrors>
</errors>
</data>
Which is the error I was getting back in the day before I implemented CookieJar. Any idea what else I can try to fix the session persistence?
Thanks!
The error stopped after 48 hours or so. It would seem that the issue was with the website's back end.