pythonseleniumfor-loopselenium-webdriverstaleelementreferenceexception

How to click all the fetched links from a search result in selenium using python?


In selenium, I am grabbing some search result URL by XPATH. Now I want to click then one by one which will open then in the same browser one by one where the base URL is opened so that I can switch between then. How can I do that? I am giving my code below.

import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By

serv_obj = Service("F:\Softwares\Selenium WebDrivers\chromedriver.exe")
driver = webdriver.Chrome(service=serv_obj)
driver.maximize_window()
driver.implicitly_wait(5)

url = "https://testautomationpractice.blogspot.com/"
driver.get(url)


driver.find_element(By.XPATH, "//input[@id='Wikipedia1_wikipedia-search-input']").send_keys("selenium")
driver.find_element(By.XPATH, "//input[@type='submit']").click()

search_result = driver.find_elements(By.XPATH, "//div[@id='wikipedia-search-result-link']/a")
links = []
for item in search_result:
    url_data = item.get_attribute("href")
    links.append(url_data)
    print(url_data)
print(len(links))
print(links)

I have grabbed all the links from the search result by using customized XPATH. I am being able yo print them also. But I want to open/click on the every resulted link one by one in the same browser.


Solution

  • You can do that as following:
    Get the list of the links.
    In a loop click on grabbed links.
    When link is opened in a new tab switch the driver to the new opened tab.
    Do there what you want to do (I simulated this by a simple delay of 1 second).
    Close the new tab.
    Switch back to the first tab.
    Collect the list of links again since the previously collected links become Stale reference.
    The following code works:

    import time
    
    from selenium import webdriver
    from selenium.webdriver.chrome.service import Service
    from selenium.webdriver.chrome.options import Options
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
    options = Options()
    options.add_argument("start-maximized")
    
    webdriver_service = Service('C:\webdrivers\chromedriver.exe')
    driver = webdriver.Chrome(options=options, service=webdriver_service)
    wait = WebDriverWait(driver, 20)
    
    
    url = "https://testautomationpractice.blogspot.com/"
    driver.get(url)
    
    wait.until(EC.element_to_be_clickable((By.XPATH, "//input[@id='Wikipedia1_wikipedia-search-input']"))).send_keys("selenium")
    wait.until(EC.element_to_be_clickable((By.XPATH, "//input[@type='submit']"))).click()
    links = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//div[@id='wikipedia-search-result-link']/a")))
    for index, link in enumerate(links):
        links[index].click()
        driver.switch_to.window(driver.window_handles[1])
        time.sleep(1)
        driver.close()
        driver.switch_to.window(driver.window_handles[0])
        links = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//div[@id='wikipedia-search-result-link']/a")))