pythonselenium-webdriverweb-scrapingxpath

Selenium - Unable to find element using xpath

I am writing a webscraper in python to get the current patch version for a list of products. I am trying to locate the text by XPATH, but I am being told that no such element exists. I have tried both an implicit and explicit wait up to 60 seconds, but still have not had any success. I see that there is an iframe on the page, but the text I am trying to access is not in that iframe, so I don't think it's relevant. Either way, I tried to switch to that iframe and find the element in there as a test, but was not successful.

This is the page I am trying to scrape

I am simply trying to get the newest version number. I have the XPATH as: /html[1]/body[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[5]/div[1]/div[1]/div[1]/div[1]/table[1]/tbody[1]/tr[2]/td[2]/span[1]

This is the code I am using

     url = "https://compatibility.rockwellautomation.com/Pages/ProductReplacement.aspx?crumb=101&restore=1&vid=53216"
        RA_XPATH = "/html[1]/body[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[5]/div[1]/div[1]/div[1]/div[1]/table[1]/tbody[1]/tr[2]/td[2]/span[1]"
        driver.get(url)
        element = WebDriverWait(driver, 60).until(EC.presence_of_all_elements_located((By.XPATH, RA_XPATH)))
        version = driver.find_element(By.XPATH, RA_XPATH)

I am able to locate it with the Rel XPath, but I would like to use the Abs XPath because it looks like it is the same across multiple products. For Example on this page, the Abs XPath is the same as the other page, but the Rel Xpath is different

I am completely stumped.
I am relatively new to all of this, so I thank you for your patience!

Solution

  • As Conal said, XPath expressions generated by the browser using the Developer Tools (F12) are not best nor optimal. You can try these as a last resource for your project. Using an Xpath expression generated on your own it will benefit your code to have a better maintainability.

    As the DOM structure (HTML tree-like code for the page) is subject to frequent changes in the future, having optimal Xpath (based on Text, Objects or Classes Names) will withstand your code to future changes. Using XPath expressions generated by the browser, will result in that your Xpath predicate will stop working.

    In regards what Xpath expression for your page. You can use this one:

    //span[contains(@class,"currentVersion")]

    Check this screenshot to get more details:

    enter image description here

    Try to learn more about Xpath, here there are 2 resources:

    https://intellipaat.com/blog/xpath-in-selenium

    https://fifithecat.medium.com/xpath-example-73bf66c2377c