pythonselenium-webdriverweb-scrapingundetected-chromedriver

THIRD_PARTY_NOTICES.chromedriver - Exec format error - undetected_chromedriver


undetected_chromedriver with webdriver_manager was working well few days ago for scraping websites but out of nowhere it started throwing the error:

OSError: [Errno 8] Exec format error: 
'/Users/pd/.wdm/drivers/chromedriver/mac64/127.0.6533.72/chromedriver-mac-x64/THIRD_PARTY_NOTICES.chromedriver'

I am guessing it is related to recent update of webdriver_manager.

This is the code:

import undetected_chromedriver as uc
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support import expected_conditions as EC

def get_driver():
    options = uc.ChromeOptions()
    # options.add_argument("--headless")
    options.add_argument("--no-sandbox")
    options.add_argument("--disable-dev-sim-usage")
    options.add_argument("--start-maximized")
    options.add_argument('--disable-popup-blocking')
    driver = uc.Chrome(driver_executable_path=ChromeDriverManager().install(), options=options, version_main=116)
    driver.maximize_window()
    return driver

It would be really great if someone can help me on this, Thanks.


Solution

  • The command ChromeDriverManager().install():

    1. creates a new folder without the executable and
    2. it retrieves the wrong file.

    enter image description here

    First, you need to remove the .wdm folder and then reinstall webdriver-manager:

    Windows Location: r"C:\Users\{user}\.wdm"

    Linux Location: /home/{user}/.wdm

    Mac Location: /Users/{user}/.wdm

    rm -rf /home/user/.wdm
    pip uninstall webdriver-manager
    pip install webdriver-manager
    

    Now, after executing ChromeDriverManager().install(), you should only see a single folder with the executable:

    enter image description here

    It check if there is really a chromedriver executable inside this folder.

    Second, it makes a correction to the file name:

    if 'THIRD_PARTY_NOTICES.chromedriver' in chromedriver_path:
        chromedriver_path = chromedriver_path.replace('THIRD_PARTY_NOTICES.chromedriver', 'chromedriver')