python-requestspython-3.6twill

Download a file using twill and python3.6


As the title suggests, i would like to download a PDF document from a website written in jspx. I've got everything working up until when i need to download a file.

This is my code

#!/usr/bin/python3

from twill.commands import *

go('http://example.com/Login.jspx')
fv('1', 'j_username', 'user')
fv('1', 'j_password', 'pass')
submit('0')

# PDF
fv('1', 'source', 'mainTempl:_id40')
submit('1')
fv('1', 'source', 'mainTempl:bill:_id224:0:_id140')
submit('0')

Now, once i do show() i can see a binary signature of a PDF file. It starts with

PDF-1.4
%����
3 0 obj
<</Intent/Perceptual/DecodeParms<</Colors 3/Predictor 15/BitsPerComponent 8/Columns 1674>>/Type/XObject/ColorSpace[/CalRGB<</Matrix[0.41239 0.21264 0.01933 0.35758 0.71517 0.11919 0.18045 0.07218 0.9504]/Gamma[2.2 2.2 2.2]/WhitePoint[0.95043 1 1.09]>>]/Subtype/Image/BitsPerComponent 8/Width 1674/Length 24551/Height 178/Filter/FlateDecode>>stream
x^��m�]�}�9��Ɨ|�x4�$�^ӎ�I�:��I��n

So i try to save it by using

redirect_output('Bill.pdf')
save_html('Bill.pdf')

It's got about the right size, but when i open it, i see two pages (it's supposed to have 2) but pages are blank! I tried modifying save_html function to save as binary (wb instead of w) but that gives me an error TypeError: a bytes-like object is required, not 'str' and i saw another method here, with using requests and saving cookies to a variable but it uses:

cookies = requests.utils.dict_from_cookiejar(get_browser()._session.cookies)

and get_browser() is not available in twill version 3. So what I'm trying is outdated. I'm not sure how to proceed from here. Any assistance is appreciated.


Solution

  • Try this code instead:

    from twill.commands import *
    
    from twill import browser
    
    go('http://example.com/Login.jspx')
    fv('1', 'j_username', 'user')
    fv('1', 'j_password', 'pass')
    submit('0')
    # PDF
    fv('1', 'source', 'mainTempl:_id40')
    submit('1')
    fv('1', 'source', 'mainTempl:bill:_id224:0:_id140')
    submit('0')
    
    with open("output.pdf", "wb") as binary_file:
        binary_file.write(browser.dump)