python-3.xseleniumselenium-webdriverweb-scrapingpageloadtimeout

How to force Selenium to refresh if the website takes too much time to load


Here's my code:

from selenium import webdriver
import time
browser = webdriver.Edge()

st1 = time.time()
browser.get("http://app1.helwan.edu.eg/Commerce/HasasnUpMlist.asp")
et1 = time.time()
el1 = et1-st1
print(f"Elapsed Time is: {el1} Seconds")
# 10.48

st2 = time.time()
my_id = browser.find_element("name","x_st_settingno")
submitting = browser.find_element("name","Submit")
my_id.send_keys(18760)
submitting.click()
et2 = time.time()
el2 = et2-st2

link_to_natega = browser.find_element("xpath",'//*[@id="ewlistmain"]/tbody/tr[3]/td[9]/font/b/span/a')
link_to_natega.click()

This works just fine. The problem is at el1. I tried making it a function to refresh using browser.refresh() when if el1>=10

But it turned out as I may have understood from the articles is that el1 is being calculated after the code is done. I want it to measure the loading time in real-time and keep updating the number so that I could keep track of and then refresh the page if the condition is met.

To state the problem clearly:

  1. The program will try to get to "http://app1.helwan.edu.eg/Commerce/HasasnUpMlist.asp"
  2. If it takes more than 10 seconds doing so, refresh(), else continue the code
  3. If el2 also is greater than 10, go back to the original URL "http://app1.helwan.edu.eg/Commerce/HasasnUpMlist.asp" and do the code again.

Solution

  • To restrict the page load time to 10 seconds you can use the set_page_load_timeout() method as follows:

    from selenium import webdriver
    from selenium.common.exceptions import TimeoutException
    
    browser = webdriver.Chrome(executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
    browser.set_page_load_timeout(10)
    while True:
        try :
            browser.get("http://app1.helwan.edu.eg/Commerce/HasasnUpMlist.asp")
            print("URL successfully Accessed")
            # continue with the next steps
        except TimeoutException as e:
            print("Page load Timeout Occurred. Refreshing !!!")
            browser.refresh()