pythonseleniumxpathcss-selectorsstaleelementreferenceexception

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document error when using Selenium and Python

I'm writing a selenium program in python to get links from a website, it runs the first time but when I re-run it, it opens the website in browser properly then it gives an error

This is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service

headers = {
    "User-Agent":
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582",
    "Accept-Language": "fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3",
    "Connection": "keep-alive",
    "Accept-Encoding": "gzip, deflate, br",
}

edge_driver = './msedgedriver.exe'
s = Service( edge_driver )
browser = webdriver.Edge( service=s )

browser.get( "https://www.jumia.ug/always/" )

all_links = browser.find_elements( By.CLASS_NAME, "core" )
working_links = []
for l in all_links:
    if l.get_attribute( "href" ) is not None:
        working_links.append(l.get_attribute('href'))

print(working_links)

this is the error i got after running it every other time

C:\Users\eliHeist\PycharmProjects\webscraping\venv\Scripts\python.exe C:/Users/eliHeist/PycharmProjects/webscraping/getlinks.py
Traceback (most recent call last):
  File "C:/Users/eliHeist/PycharmProjects/webscraping/getlinks.py", line 34, in <module>
    if l.get_attribute( "href" ) is not None:
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 155, in get_attribute
    attribute_value = self.parent.execute_script(
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 878, in execute_script
    return self.execute(command, {
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: MicrosoftEdge=96.0.1054.57)
Stacktrace:
Backtrace:
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF60587CB62+56946]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AA597+949863]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AD788+962648]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AE3EA+965818]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605516D9F+1394287]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605501546+1306134]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605515D5D+1390125]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6055013BF+1305743]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054D6AA9+1131385]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054D7B8F+1135711]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF60556C2E5+1743797]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF60556A771+1736769]
    Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF60574B519+2521]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6055420AE+1571198]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF60587576C+27260]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF6058750E4+25588]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF605874F36+25158]
    Microsoft::Applications::Events::EventProperties::GetName [0x00007FF6057D02DC+211724]
    BaseThreadInitThunk [0x00007FFF89E254E0+16]
    RtlUserThreadStart [0x00007FFF8AF2485B+43]


Process finished with exit code 1

What should I be changing or not doing, the page does not refresh at all?

Solution

  • To print the value of the href attribute you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies: