pythonselenium-webdriverautomationselenium-chromedriverbots

Selenium WebDriver Click Leads to White Page Instead of Expected Content

I am using Selenium WebDriver with Chrome to automate interactions on a website. My script successfully clicks a button on the page, which is expected to navigate to a new page. However, instead of showing the expected content, the browser displays a white page. But i am sure that it clicks the correct button because after the click, the website link changes to the one of the new page it is supposed to go to. The only problem is that the screen stays white.

I have purposly used the Xpath of the button just to make sure it will click the correct button.

there must be a problem with the site i think. Maybe it detects bots. image of the white page

but when I go directly in the script to the desired page jsut by filling in the link in driver.get() it works and does not show a white page.

here is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

driver = webdriver.Chrome()

driver.maximize_window()
driver.get("https://www.lebara.nl/nl/prepaid/data-bundle-valuesim.html")
time.sleep(1)

cookieDeclineButton = driver.find_element(By.ID, "onetrust-reject-all-handler")
cookieDeclineButton.click()
time.sleep(5)

bestelSimkaartButton = driver.find_element(By.XPATH, "/html/body/div[2]/div[1]/div[1]/div/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/div[2]/div[3]/button")
bestelSimkaartButton.click()
time.sleep(5)

input("Enter key: ")

i have used all kind of locators(ID's , CSS selectors etc) but i stick for the moment with the Xpath.

Solution

  • it’s likely that this website has more sophisticated bot detection mechanisms in place. Here are some additional advanced techniques you can try to bypass or work around these detections:

    1. Use Undetected WebDriver: Some websites specifically detect Selenium WebDriver. Libraries like undetected-chromedriver can help make your script less detectable. Install the library:

    pip install undetected-chromedriver

    2. Simulate Human-Like Behavior: Websites often detect bots based on behavior. Adding random delays and mouse movements can make your script appear more human-like. Install the pyautogui library for mouse movements:

    pip install pyautogui

    3. Handle JavaScript Challenges: Some websites use JavaScript challenges to detect bots. You can try executing JavaScript directly to bypass these challenges.

    Here's a working solution for the issue you're facing. let me know if this works for you as well.

    import time
import random
import pyautogui
import undetected_chromedriver as uc
from selenium.webdriver.common.by import By

# Use undetected-chromedriver
driver = uc.Chrome()
driver.maximize_window()
driver.get("https://www.lebara.nl/nl/prepaid/data-bundle-valuesim.html")

# Simulate human-like behavior
time.sleep(random.uniform(1, 3))
pyautogui.moveTo(random.randint(100, 500), random.randint(100, 500), duration=0.5)

# Click the cookie decline button
cookie_decline_button = driver.find_element(By.ID, "onetrust-reject-all-handler")
cookie_decline_button.click()

# Simulate human-like behavior
time.sleep(random.uniform(1, 3))
pyautogui.moveTo(random.randint(100, 500), random.randint(100, 500), duration=0.5)

# Click the bestel_simkaart_button using JavaScript
bestel_simkaart_button = driver.find_element(By.XPATH, "/html/body/div[2]/div[1]/div[1]/div/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/div[2]/div[3]/button")
driver.execute_script("arguments[0].click();", bestel_simkaart_button)

# Wait for the new page to load
time.sleep(5)