I have a web page which loads content dynamically and while page loads, there is spinning wheel, I already found solution to grab content loaded immediately on page, but seems i can't find solution to grab content loaded later in dom.
What i can think of is to find element with specific class of that wheel spinning, and wait for it to change, once it's changed, than it means content is loaded in dom.
I am using Selenium
with Firefox
webdriver
on Ubuntu
.
Here is the class i am looking to monitor:
<div class="wheel spinning"></div>
Once content is loaded, wheel stop spinning and class is changed to:
<div class="wheel"></div>
Anyone find solution to find and monitor class="wheel spinning"
and once it's changed to class="wheel"
to continue to grab data.
Edit:
The XPATH actually solved one part of solution, here's part of code
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//*[@class='wheel']))
)
title = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[3]')
print(title.text)
But if element don't appear within 10 seconds it error's out, now to find a way to retry again and again until element is present on page.
Is there a difference in use presence_of_element_located((By.XPATH))
and find_element_by_xpath
@LucasTierney's answer was in the right direction. However I still feel the solution can be optimized as follows:
As the wheel is visible, instead of presence_of_element_located()
method you need to use visibility_of_element_located()
method.
The node:
<div class="wheel spinning"></div>
Can't be located through the XPath containing a single class i.e. only wheel
as in:
el = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, "//*[@class='wheel']")))
Instead you can use either of the Locator Strategies:
cssSelector
:
el = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.wheel.spinning")))
WebDriverWait(driver, 10).until(lambda d: 'spinning' not in el.get_attribute('class'))
xpath
:
el = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='wheel spinning']")))
WebDriverWait(driver, 10).until(lambda d: 'spinning' not in el.get_attribute('class'))