pythonpython-3.xurldownloadmechanicalsoup

MechanicalSoup with Login: Unable to browser.download_link() with given URL & save-path


Im currently trying to fetch some files from an internal network. I managed to fetch the URLs of some files in the format https://example.site/files/testfile.pdf. Now I've tried to download this particular file by using the following:

import mechanicalsoup

browser = mechanicalsoup.StatefulBrowser()
for s in site_links ##s = 'https://example.site/file/.../.../file.pdf'
##print(s)
browser.download_link(s, "X:/FolderA/FolderB/test.pdf")

But each time PyCharm throws an error at me which I don't understand (I am new to Python).

I've searched around but all I found was Python2 related stuff or downloads without authentication with username and password.


Solution

  • In your code snippet, s is a URL; but browser.download_link takes a link, i.e. something like <a href="...">...</a>.

    You can pass download_link the link if you have it. If not, MechanicalSoup doesn't have a convenience function for downloading from a URL directly, so you would need to do a little more work. For example, the following (untested) code should do what you want:

    browser = mechanicalsoup.StatefulBrowser()
    response = browser.open(url)
    with open('file_to_download_to', 'wb') as fid:
        fid.write(response.content)