pythonpython-3.xauthenticationmechanicalsoupstay-logged-in

Python: Remain logged in a website using mechanicalsoup

I am a newbie in Python. I want to write a script in python where I run thousands of URLs and save the response. In order to access these URLs, credentials are required. So, I have written a basic script which goes to the URL and print the response. When I go through multiple URL, the website returns the error that multiple users are logged in. So, I would like to login once and run other URLs in the same logged in session. Is there any way I can do that using Mechanical soup.

Here is my script:

import mechanicalsoup

browser = mechanicalsoup.StatefulBrowser()
browser.open("mywebsite1")

browser.select_form('form[action="/login"]')
browser.get_current_form().print_summary()
browser["userId"] = "myusername"
browser["password"] = "mypassword"

response = browser.submit_selected()
browser.launch_browser()
print(response.text)
browser.open("mywebsite2")
print(response.text)
...... so on for all the URLs

How can I save my session? Thanks in advance for your help

Solution

  • Just save & load session.cookies:

    def save_cookies(browser):
    return browser.session.cookies.get_dict()

def load_cookies(browser, cookies):
    from requests.utils import cookiejar_from_dict
    browser.session.cookies = cookiejar_from_dict(cookies)

browser = mechanicalsoup.StatefulBrowser()
browser.open("www.google.com")

cookies = save_cookies(browser)
load_cookies(browser, cookies)