Attempting to select a <textarea> on a website and submit it? (Using Python Mechanize)

I am trying to submit the textarea on https://keywordsheeter.com/ (name="findersearchQueryInput") then I want to be able to print the results of the submitted textarea to my console.

I tried form.set_value, form.find_control, and form["something"] = "something".

I realized this isn't working because the initial select_form() isn't finding the name (findersearchQueryInput) which is the name of the textarea on keywordsheeter.com

I was expecting to be able to find the form and go from there.

This is the code I have so far:

import mechanize

br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [('User-agent',
                  'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
br.open("https://keywordsheeter.com/")

print(br.title())

print(br.forms())

br.select_form("findersearchQueryInput")  # < not working...

br["keto diet "] = ["findersearchQueryInput"]

br.submit()

print(br.form)

Thanks so much in advance!

Solution

You should use selenium because Mechanize does not support JavaScript

first install it pip install selenium then follow the instruction get the appropriate webDriver https://sites.google.com/a/chromium.org/chromedriver/downloads

Then you can modify your code like that:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

webdriver_path = "path/to/chromedriver"

browser = webdriver.Chrome(executable_path=webdriver_path)
browser.get("https://keywordsheeter.com/")

# Find the textarea and enter your text
textarea = browser.find_element_by_name("findersearchQueryInput")
textarea.send_keys("keto diet")

# Submit the form by pressing Enter
textarea.send_keys(Keys.RETURN)

# Wait for the results to load
wait = WebDriverWait(browser, 10)
results = wait.until(EC.presence_of_element_located((By.CLASS_NAME, "result-list")))

# Print the results
print(results.text)

# Close the browser
browser.quit()