I'm trying some stuff out with selenium, and I really want my script to run quickly.
I thought that running my script with headless Chrome would make it faster.
First, is that assumption correct, or does it not matter if I run my script with a headless driver?
I want headless Chrome to work, but somehow it isn't working correctly. I tried different things, and most suggested that it would work as said here in the October update:
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
But when I tried that, I saw weird console output, and it still doesn't seem to work.
Any tips appreciated.
To run chrome-headless just add --headless via chrome_options.add_argument, e.g.:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
# chrome_options.add_argument("--disable-extensions")
# chrome_options.add_argument("--disable-gpu")
# chrome_options.add_argument("--no-sandbox") # linux only
chrome_options.add_argument("--headless=new") # for Chrome >= 109
# chrome_options.add_argument("--headless")
# chrome_options.headless = True # also works
driver = webdriver.Chrome(options=chrome_options)
start_url = "https://duckgo.com"
driver.get(start_url)
print(driver.page_source.encode("utf-8"))
# b'<!DOCTYPE html><html xmlns="http://www....
driver.quit()
So my thought is that running it with headless chrome would make my script faster.
Try using chrome options like --disable-extensions or --disable-gpu and benchmark it, but I wouldn't count with substantial improvement.
References: headless-chrome