pythonseleniumselenium-webdriverdrop-down-menuironwebscraper

Unable to locate or select a select element from dropdown menu with Selenium - element not visible

edited:

[https://www.bellsofsteel.us/checkout/][1]

I'm unable to locate or select an option from a drop-down menu using selenium. I'm attempting to get the various taxes and shipping for an item, by iterating through a list with cities, states and zip codes

The HTML for the select element is as follows:

<span class="woocommerce-input-wrapper">
    <select name="billing_state" id="billing_state" class="state_select select2-hidden-accessible"        autocomplete="address-level1" data-placeholder="State" data-input-classes="" data-label="State /     County" tabindex="-1" aria-hidden="true">
    <option value="">Select an option…</option>
    <option value="AL">Alabama</option>
    <option value="AZ">Arizona</option>
    <option value="AR">Arkansas</option>
    <option value="CA">California</option>
    .........
    <option value="WY">Wyoming</option>
    </select>

    <span class="select2 select2-container select2-container--default select2-container--above select2-container--open" dir="ltr" style="width: 100%;">
       <span class="selection">
         <span class="select2-selection select2-selection--single" aria-haspopup="true" aria-expanded="true" tabindex="0" aria-label="State / County" role="combobox" aria-owns="select2-billing_state-results" aria-activedescendant="select2-billing_state-result-og2a-AR">
        <span class="select2-selection__rendered" id="select2-billing_state-container" role="textbox" aria-readonly="true">
          <span class="select2-selection__placeholder">State</span> 
        </span>
        <span class="select2-selection__arrow" role="presentation">
           <b role="presentation"></b>
        </span>
         </span>
      </span>
    <span class="dropdown-wrapper" aria-hidden="true">
    </span>
</span>

I've tried this:

dropdown1 = Select(driver.find_element(By.CSS_SELECTOR, 'select[name="billing_state"]'))
dropdown1.select_by_visible_text('California')

Which returns the error:

Message: element not interactable: Element is not currently visible and may not be manipulated

I've also used Expected Conditions

element_present = EC.text_to_be_present_in_element((By.CSS_SELECTOR,'select[name="billing_state"]'),item[1])
                WebDriverWait(driver, 20).until(element_present)

Which will just time out.

I can select the actual drop down menu with

element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, 'span.selection')
WebDriverWait(driver, 30).until(element_present)
driver.find_element(By.CSS_SELECTOR, 'span.selection').click()

which will open the drop-down menu but not make the elements clickable

Any help would be greatly appreaciated!

Posted some of the code I've been using here are the things I've tried:

###This will click on the drop down menu so that you can see it open in the selenium window:
element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, 'span.selection'))
WebDriverWait(driver, 30).until(element_present)
try:
    driver.find_element(By.CSS_SELECTOR, 'span.selection').click()
except:
    clicker = driver.find_element(By.CSS_SELECTOR, 'span.selection')
    driver.execute_script("arguments[0].click();", clicker)

##This attempts to select from the select options:

dropdown1 = Select(driver.find_element(By.CSS_SELECTOR, 'select[name="billing_state"]'))
dropdown1.select_by_visible_text('California')

essentially the same thing but with the span class directly proceeding the select element (receives click but does not drop down menu)

###This will click on the drop down menu so that you can see it open in the selenium window:
element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, 'span.woocommerce-input-wrapper'))
WebDriverWait(driver, 30).until(element_present)
try:
    driver.find_element(By.CSS_SELECTOR, 'span.woocommerce-input-wrapper').click()
except:
    clicker = driver.find_element(By.CSS_SELECTOR, 'span.woocommerce-input-wrapper')
    driver.execute_script("arguments[0].click();", clicker)

##This attempts to select from the select options:

dropdown1 = Select(driver.find_element(By.CSS_SELECTOR, 'select[name="billing_state"]'))
dropdown1.select_by_visible_text('California')

Both return the following:

Message: element not interactable: Element is not currently visible and may not be manipulated

When I manually click on the option I can see the HTML change from:

<span class="select2-selection__rendered" id="select2-billing_state-container" role="textbox" aria-readonly="true">
 <span class="select2-selection__placeholder">State</span>

To

<span class="select2-selection__rendered" id="select2-billing_state-container" role="textbox" aria-readonly="true" title="California">California</span>

I thought I might be able to f string literal the title like so:


element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, 'span.select2-selection__rendered'))
WebDriverWait(driver, 30).until(element_present)
driver.find_element(By.CSS_SELECTOR, 'span.select2-selection__rendered').click()


element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, element_present_click(f'span[title=\"California\"]'))
WebDriverWait(driver, 30).until(element_present)
driver.find_element(By.CSS_SELECTOR, element_present_click(f'span[title=\"California\"]').click()

but it also timed out as well.

Also tried to select by value:

driver.find_element(By.CSS_SELECTOR, 'span.selection').click()
dropdown1 = Select(driver.find_element(By.CSS_SELECTOR, 'select[name="billing_state"]'))
dropdown1.select_by_value('CA')

Same thing - element not interactable

Solution

  • That dropdown is not a 'Select' type element, it is a 'ul' type element, so you can't use Select.

    In the checkout page, add the below code and try:

    # scrolling to the element - 'First name' label
first_name_label = driver.find_element(By.XPATH, ".//label[@for='billing_first_name']")
driver.execute_script("arguments[0].scrollIntoView(true)", first_name_label)

# clicking on the 'State / County' dropdown
driver.find_element(By.XPATH, "(.//*[@aria-label='State / County'])[1]").click()
sleep(1)
# getting the list of all the states
list_of_states = driver.find_elements(By.CSS_SELECTOR, "#select2-billing_state-results li")

# state name to be selected
state_to_select = "South Dakota"
i = 0

# select the state
for state in list_of_states:
    if state.text == state_to_select:
        driver.find_element(By.XPATH, ".//ul[@id='select2-billing_state-results']/li[" + str(i + 1) + "]").click()
        break
    i += 1