I'm using Python with Selenium 4.25.0 to automate Google Chrome 136. My goal is to have Selenium use my existing, logged-in "Default" Chrome profile to navigate to a specific URL (https://aistudio.google.com/prompts/new_chat) and perform actions.
The Problem:
When I execute my script:
Chrome launches successfully.
It clearly loads my "Default" profile, as I see my personalized "New Tab" page (title: "新分頁", URL: chrome://newtab/) with my usual shortcuts, bookmarks bar, and theme. This confirms the --user-data-dir and --profile-directory arguments are pointing to the correct profile.
However, the subsequent driver.get("https://aistudio.google.com/prompts/new_chat") command does not navigate the browser. The browser remains on the chrome://newtab/ page.
I am very diligent about ensuring all chrome.exe processes are terminated (checked via Task Manager) before running the script.
Environment:
OS: Windows 11 Pro, Version 23H2
Python Version: 3.12.x
Selenium Version: 4.25.0
Chrome Browser Version: 136.0.7103.93 (Official Build) (64-bit)
ChromeDriver Version: 136.0.7103.xx (downloaded from official "Chrome for Testing" site for win64, matching Chrome's major.minor.build)
Simplified Code Snippet:
import time
import os # For path checks
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import WebDriverException
# --- Configuration ---
# User needs to replace CHROME_DRIVER_PATH with the full path to their chromedriver.exe
CHROME_DRIVER_PATH = r'C:\path\to\your\chromedriver-win64\chromedriver.exe'
# User needs to replace YourUserName with their actual Windows username
CHROME_PROFILE_USER_DATA_DIR = r'C:\Users\YourUserName\AppData\Local\Google\Chrome\User Data'
CHROME_PROFILE_DIRECTORY_NAME = "Default" # Using the standard default profile
TARGET_URL = "https://aistudio.google.com/prompts/new_chat" # Example
def setup_driver():
print(f"Driver Path: {CHROME_DRIVER_PATH}")
print(f"User Data Dir: {CHROME_PROFILE_USER_DATA_DIR}")
print(f"Profile Dir Name: {CHROME_PROFILE_DIRECTORY_NAME}")
if not os.path.exists(CHROME_DRIVER_PATH):
print(f"FATAL: ChromeDriver not found at '{CHROME_DRIVER_PATH}'.")
return None
if not os.path.isdir(CHROME_PROFILE_USER_DATA_DIR):
print(f"FATAL: Chrome User Data dir not found at '{CHROME_PROFILE_USER_DATA_DIR}'.")
return None
chrome_options = Options()
chrome_options.add_argument(f"--user-data-dir={CHROME_PROFILE_USER_DATA_DIR}")
chrome_options.add_argument(f"--profile-directory={CHROME_PROFILE_DIRECTORY_NAME}")
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation", "load-extension"])
chrome_options.add_experimental_option('useAutomationExtension', False)
# chrome_options.add_argument("--disable-blink-features=AutomationControlled") # Tried with and without
chrome_options.add_argument("--start-maximized")
try:
service = Service(executable_path=CHROME_DRIVER_PATH)
driver = webdriver.Chrome(service=service, options=chrome_options)
print("WebDriver initialized.")
return driver
except WebDriverException as e:
print(f"FATAL WebDriverException during setup: {e}")
if "user data directory is already in use" in str(e).lower():
print(">>> Ensure ALL Chrome instances are closed via Task Manager.")
return None
except Exception as e_setup:
print(f"Unexpected FATAL error during setup: {e_setup}")
return None
def main():
print("IMPORTANT: Ensure ALL Google Chrome instances are FULLY CLOSED before running this script.")
input("Press Enter to confirm and continue...")
driver = setup_driver()
if not driver:
print("Driver setup failed. Exiting.")
return
try:
print(f"Browser launched. Waiting a few seconds for it to settle...")
print(f"Initial URL: '{driver.current_url}', Initial Title: '{driver.title}'")
time.sleep(4) # Increased wait after launch for profile to fully 'settle'
print(f"Attempting to navigate to: {TARGET_URL}")
driver.get(TARGET_URL)
print(f"Called driver.get(). Waiting for navigation...")
time.sleep(7) # Increased wait after .get() for navigation attempt
current_url_after_get = driver.current_url
current_title_after_get = driver.title
print(f"After 7s wait - Current URL: '{current_url_after_get}', Title: '{current_title_after_get}'")
if TARGET_URL not in current_url_after_get:
print(f"NAVIGATION FAILED: Browser did not navigate to '{TARGET_URL}'. It's still on '{current_url_after_get}'.")
# Could also try JavaScript navigation here for more info
# print("Attempting JavaScript navigation as a fallback test...")
# driver.execute_script(f"window.location.href='{TARGET_URL}';")
# time.sleep(7)
# print(f"After JS nav attempt - URL: '{driver.current_url}', Title: '{driver.title}'")
else:
print(f"NAVIGATION SUCCESSFUL to: {current_url_after_get}")
except Exception as e:
print(f"An error occurred during main execution: {e}")
finally:
print("Script execution finished or errored.")
input("Browser will remain open for inspection. Press Enter to close...")
if driver:
driver.quit()
if __name__ == "__main__":
# Remind user to update paths if placeholders are detected
if r"C:\path\to\your\chromedriver-win64\chromedriver.exe" == CHROME_DRIVER_PATH or \
r"C:\Users\YourUserName\AppData\Local\Google\Chrome\User Data" == CHROME_PROFILE_USER_DATA_DIR:
print("ERROR: Default placeholder paths are still in the script.")
print("Please update CHROME_DRIVER_PATH and CHROME_PROFILE_USER_DATA_DIR with your actual system paths.")
else:
main()
Console Output (when it gets stuck on New Tab):
Setting up Chrome driver from: C:\Users\stat\Downloads\chromedriver-win64\chromedriver-win64\chromedriver.exe
Attempting to use Chrome User Data directory: C:\Users\stat\AppData\Local\Google\Chrome\User Data
Attempting to use Chrome Profile directory name: Default
WebDriver initialized successfully.
Browser launched. Waiting a few seconds for it to settle...
Initial URL: 'chrome://newtab/', Initial Title: '新分頁'
DevTools remote debugging requires a non-default data directory. Specify this using --user-data-dir.
[... some GCM / fm_registration_token_uploader errors may appear here ...]
Attempting to navigate to: https://aistudio.google.com/prompts/new_chat
Called driver.get(). Waiting for navigation...
After 7s wait - Current URL: 'chrome://newtab/', Title: '新分頁'
NAVIGATION FAILED: Browser did not navigate to 'https://aistudio.google.com/prompts/new_chat'. It's still on 'chrome://newtab/'.
What I've Already Verified/Tried:
ChromeDriver version precisely matches the Chrome browser's major.minor.build version (136.0.7103).
All chrome.exe processes are terminated via Task Manager before script execution.
The paths for CHROME_DRIVER_PATH, CHROME_PROFILE_USER_DATA_DIR, and CHROME_PROFILE_DIRECTORY_NAME are correct for my system.
The browser visibly loads my "Default" profile (shows my theme, new tab page shortcuts).
Tried various time.sleep() delays.
The "DevTools remote debugging requires a non-default data directory" warning appears, as do some GCM errors, but the browser itself opens with the profile.
My Question:
Given that Selenium successfully launches Chrome using my specified "Default" profile (as evidenced by my personalized New Tab page loading), why would driver.get() fail to navigate away from chrome://newtab/? Are there specific Chrome options for Selenium 4.25+ or known issues with Chrome 136 that could cause this behavior when using an existing, rich user profile, even when Chrome is fully closed beforehand? How can I reliably make driver.get() take precedence over the default New Tab page loading in this scenario?
The root cause could be that the ChromeDriver (≥ v113 with “Chrome for Testing”) intentionally limits automation on “default” or regular profiles for security and stability. This is reflected in the warning:
"DevTools remote debugging requires a non-default data directory"
This means:
ChromeDriver can't fully control Chrome if you use --user-data-dir pointing to Chrome's real profile directory.
Navigation via driver.get() fails silently or gets overridden by the "New Tab" logic in Chrome itself.
Even though Chrome opens with the correct theme/profile, the DevTools protocol is not fully attached, so
driver.get()doesn't execute properly.
To fix this issue, you can use a dedicated test profile instead of Default
Create a copy of your "Default" profile into a separate folder (e.g., ChromeProfileForSelenium) and point --user-data-dir there without specifying the --profile-directory.
create a copy
mkdir "C:\SeleniumChromeProfile"
xcopy /E /I "%LOCALAPPDATA%\Google\Chrome\User Data\Default" "C:\SeleniumChromeProfile\Default"
and then update your script:
chrome_options.add_argument("--user-data-dir=C:\\SeleniumChromeProfile")
# Omit this line: chrome_options.add_argument("--profile-directory=Default")
This avoids Chrome's protections around Default, and lets driver.get() work as expected.