pythonurllib2mechanizeurllibtwill

Python: Clicking a button with urllib or urllib2


I want to click a button with python, the info for the form is automatically filled by the webpage. the HTML code for sending a request to the button is:

INPUT type="submit" value="Place a Bid">

How would I go about doing this? Is it possible to click the button with just urllib or urllib2? Or will I need to use something like mechanize or twill?


Solution

  • Use the form target and send any input as post data like this:

    <form target="http://mysite.com/blah.php" method="GET">
        ......
        ......
        ......
        <input type="text" name="in1" value="abc">
        <INPUT type="submit" value="Place a Bid">
    </form>
    

    Python:

    # parse the page HTML with the form to get the form target and any input names and values... (except for a submit and reset button)
    # You can use XML.dom.minidom or htmlparser
    # form_target gets parsed into "http://mysite.com/blah.php"
    # input1_name gets parsed into "in1"
    # input1_value gets parsed into "abc"
    
    form_url = form_target + "?" + input1_name + "=" + input1_value
    # form_url value is "http://mysite.com/blah.php?in1=abc"
    
    # Then open the new URL which is the same as clicking the submit button
    s = urllib2.urlopen(form_url)
    

    You can parse the HTML with HTMLParser

    And don't forget to urlencode any post data with:

    urllib.urlencode(query)