I am trying to use Selenium in order to learn different ways of web scraping.
When the code is executed Firefox starts and the "accept cookies" or what ever pops up. I am unable to locate the "accept" button when inspecting the page.
my code so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
import pandas as pd
import time
PATH = "C:/Users/myuser/Desktop/Driver/geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.maximize_window() # For maximizing window
driver.get("https://www.immonet.de/")
button_pos = driver.find_element(by=By.CLASS_NAME, value="sc-gsDKAQ fILFKg")
button_pos.click()
print(driver.title)
input = input()
I get the following error: Unable to locate element: .sc-gsDKAQ fILFKg
My thought was locating the button via the inspect tool as follows:
What am I missing or doing wrong? How would i find the right element?
Thanks! Pat
First of all, to display this url,accepting the cookies is a must but to accept and click on the cookie button isn't a easy task because cookies button is under shadow root (open) selenium and webdriverWait can do nothing on shadow root,so to execute shadow root you need to apply JavaScript querySelector.
#To execute shadow root and accept cookies
driver.execute_script('''return document.querySelector('div#usercentrics-root').shadowRoot.querySelector('button[data-testid="uc-accept-all-button"]')''').click()