pythonseleniumselenium-webdrivercss-selectorsironwebscraper

Looking for a way to capture the updated text of an element after input with Selenium

I've been creating a webscraper with selenium that will select several items, fill a cart, and then cycle through a list of zip codes, cities, and states to calculate the total cost + shipping + tax for a variety of locations.

The scraper has been set up successfully and works, but the problem that I am running into is that the text of the element I am searching for gives the base price, and then updates to include tax and shipping. I've tried using implicit wait or time.sleep() to slow the process down to give it time to update, but it continues to only get the original price. Without putting in the entire script this is what I am running into:

 for item in c_s_z_list:
                cty = driver.find_element(By.CSS_SELECTOR, 'input[id="city"]')
                cty.clear()
                time.sleep(1)
                cty.send_keys(item[0]) 
                
                
                dropdown1 = Select(driver.find_element(By.CSS_SELECTOR,'select[id="state"]'))
                dropdown1.select_by_visible_text(item[1])
                
                zcode = driver.find_element(By.CSS_SELECTOR,'input[id="zip"]')
                zcode.clear()
                time.sleep(1)
                zcode.send_keys(item[2]) 
                
                
                time.sleep(20)
                
                
                fpt = driver.find_element(By.CSS_SELECTOR,'span[class="order-wizard-cart-summary-grid-right"]')
                fp = fpt.text
                print(fp)

Before entering in the city, state and zip info the html looks like this:

enter image description here

and after the info is added it updates to this:

enter image description here

Even after giving enough sleep for the page to load, the element.text always returns the initial value, not the updated one.

I've played around with waits, and sleeps and even refreshing to see if any of that makes a difference, but so far no dice

This is the main landing page

https://www.performbetter.com/First-Place-Gravity-Kettlebell

And this is the cart specific page

https://checkout.performbetter.com/sca-dev-vinson/checkout.ssp?is=checkout&fragment=opc&_ga=2.53878443.2036683866.1667331254-1479525696.1666912626#opc

Solution

  • There are two matches for your locator value, so it returns the SubTotal, change locator value to:

    driver.find_element(By.CSS_SELECTOR,'.order-wizard-cart-summary-total .order-wizard-cart-summary-grid-right')