linuxseleniumpageloadpage-load-timepageloadtimeout

Make Selenium continue after a timeout error


I have a code that returns the title of a list of websites. Sometimes, a website takes an absurd amount of time to load, so when that happens, a timeout error prompts. I'd like to make it so that when such an error happens, the program continues running as opposed to stopping completely.

The code is:

from pyvirtualdisplay import Display
from time import sleep
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
display = Display(visible=0, size(800,600))
display.start()
driver = webdriver.Firefox(executable_path='/usr/local/lib/geckodriver/geckodriver')
driver.set_page_load_timeout(60)
driver.get('https://google.com')
print(driver.title)

The following code is what makes a timeout occur when after 60 seconds the page hasn't loaded:

driver.set_page_load_timeout(60)

When the 60 seconds pass, the program stops and prompts the timeout error. I want it to continue with the next url instead.


Solution

  • To iterate over a List of urls even incase of page_load_timeout error, you can use the following solution:

    You can find a detailed discussion on page load timeout in How to set the timeout of 'driver.get' for python selenium 3.8.0?