pythonselenium-webdriverweb-scraping

How to disable selenium logs AND run the browser in headless mode


This is my code as of now:

from selenium import webdriver

from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])

if not("-dhl" in sys.argv or "--disable-headless" in sys.argv):
    options.add_argument("--headless=new")

self.driver = webdriver.Chrome(
    service=Service(ChromeDriverManager().install()),
    options=options
)

driver.get("https://google.com")

The problem is only 1 of them will work at the same time, meaning if I try to run the browser without headless mode (run with the --disable-headless flag) the logs are turned off but if I don't do that the browser will run in headless mode but still display logs.

I've already tried using options.headless = True, disabling logs through logging module and setting it to logging.CRITICAL through the logging module and even through the selenium logging, changing the order of these 2 sections to ensure the settings aren't being overwritten but none of them seems to be the issue.

In short, I can't do both things simultaneously for some unknown reason.


Solution

  • You need to direct ChromeDriver logs to os.devnull:

    service = Service(
        ChromeDriverManager().install(),
        log_path=os.devnull
    )
    

    Then suppress the urllib3, selenium, WDM, and webdriver-manager logs:

    logging.getLogger("selenium").setLevel(logging.CRITICAL)
    logging.getLogger("urllib3").setLevel(logging.CRITICAL)
    logging.getLogger("WDM").setLevel(logging.CRITICAL)
    logging.getLogger("webdriver_manager").setLevel(logging.CRITICAL)
    

    Also turn down Chrome's logging

    options.add_argument("--log-level=3")
    

    This should help!