Python 3.11
Selenium 4.38
For the following URL:
https://www.amazon.es/-/en/Womens-Lace-up-Comfortable-Anti-Slip-SDMB2203W/dp/B0B5CX9VZY
On the right hand side of the page it says "Dispatches From" and "Amazon".
I'm writing a Selenium script to try to find pages where that field doesn't say Amazon, however Selenium cannot find that element in order to read it.
Class: a-size-small offer-display-feature-text-message << that space in 'small offer' causes a compound class name error, which does not resolve by inserting a period.
CSS Selector: #fulfillerInfoFeature_feature_div > div.offer-display-feature-text.a-size-small > div.offer-display-feature-text.a-spacing-none.odf-truncation-popover > span << does not find element.
XPath: '//\*\[@id="fulfillerInfoFeature_feature_div"\]/div\[2\]/div\[1\]/span' << does not find element.
Full XPath: /html/body/div\[1\]/div\[1\]/div/div\[2\]/div\[2\]/div\[2\]/div/div/div\[2\]/div\[5\]/div/div\[1\]/div/div/div/form/div/div/div/div/div\[4\]/div/div\[21\]/div/div/div\[1\]/div/div\[2\]/div\[2\]/div\[1\]/span << does not find element.
There doesn't appear to be a relevant iframe in play (there are iframes but I don't see that they apply to this element), and I don't see any shadow DOM.
How can I grab this element via Selenium?
EDIT: Request for more code. Below are many of the things I have tried:
dispatch = driver.find_element(By.CLASS_NAME,'a-size-small offer-display-feature-text-message')
dispatch = driver.find_element(By.CLASS_NAME,"span.a-size-small.offer-display-feature-text-message")
dispatch = driver.find_element(By.CLASS_NAME, "'a-size-small offer-display-feature-text-message'")
dispatch = driver.find_element(By.CSS_SELECTOR,"#fulfillerInfoFeature_feature_div > div.offer-display-feature-text.a-size-small > div.offer-display-feature-text.a-spacing-none.odf-truncation-popover > span")
dispatch = driver.find_element(By.XPATH,"//span[contains(text(), 'Amazon')]")
dispatch = driver.find_element(By.CSS_SELECTOR,"span.div.offer-display-feature-text.a-spacing-none.odf-truncation-popover")
dispatch = driver.find_element(By.XPATH,'//*[@id="fulfillerInfoFeature_feature_div"]/div[2]/div[1]/span')
dispatch = driver.find_element(By.XPATH,'//*[@id="merchantInfoFeature_feature_div"]/div[2]/div[1]/span')
dispatch = driver.find_element(By.XPATH,"/html/body/div[1]/div[1]/div/div[2]/div[2]/div[2]/div/div/div[2]/div[5]/div/div[1]/div/div/div/form/div/div/div/div/div[4]/div/div[21]/div/div/div[1]/div/div[2]/div[2]/div[1]/span")
EDIT 2: My comments button on answers isn't working (I think it's a low karma thing) - to respond to @Corey Goldberg, I tried both of those suggestions and got the associated messages:
.a-size-small.offer-display-feature-text-message
Message: invalid selector: An invalid or illegal selector was specified
a-size-small offer-display-feature-text-message
Message: Compound class names are not allowed
Here are the answers to your questions. The working code is posted at the bottom.
Compound class name error
The class below,
a-size-small offer-display-feature-text-message
is actually two classes. The space separates the two classes, a-size-small and offer-display-feature-text-message.
When you use By.CLASS_NAME(), it expects a single class. If you use that entire string, it will throw, invalid selector: Compound class names not permitted. To fix this, you can either choose one of the classes to use (assuming the element can be uniquely found with only one class) or convert it to a CSS selector, .a-size-small.offer-display-feature-text-message.
Having said that, this locator doesn't work because it actually returns 7 different elements so we need to find a better locator.
The CSS selector, #fulfillerInfoFeature_feature_div > div.offer-display-feature-text.a-size-small > div.offer-display-feature-text.a-spacing-none.odf-truncation-popover > span doesn't find the element
When I tested it in the browser, it does find the element. My guess is that you aren't using a proper wait and that's why it's not found... the page is still loading in the background. A proper wait would be a WebDriverWait, as below.
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located(locator))
Having said that, that locator is excessively long and can be simplified.
The XPath, //*[@id='fulfillerInfoFeature_feature_div']/div[2]/div[1]/span doesn't find the element
When I tested it in the browser, it works but I had to clean up all the escaping you had in the string first. The XPath string above, as written, works.
Absolute XPath... what you called, "full XPath"
Don't use absolute XPaths because they are very brittle... meaning the slightest change in the DOM will break your locator.
IFRAME or shadow DOM
Neither are in play here.
I think the main issue is that you aren't using a proper wait.
NOTE: The product you linked is showing as out of stock for me so it does not have the "Dispatches from Amazon" section. I chose another product linked from the original and it's working.
Here's what I found that works.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
url = 'https://www.amazon.es/-/en/G-STAR-Womens-Skinny-Destroyed-D19079-8971-c267/dp/B08KS8FQ6N'
driver = webdriver.Chrome()
driver.maximize_window()
driver.get(url)
wait = WebDriverWait(driver, 10)
dispatch = wait.until(EC.visibility_of_any_elements_located((By.CSS_SELECTOR, "#fulfillerInfoFeature_feature_div span")))[1].text
print(f"Dispatch info: {dispatch}")
It prints
Amazon