I'm trying to use selenium to scrape data that requires you to push on each round to display more data, but I'm quite inexperienced with selenium and having trouble locating the element to scrape the data from
I'm using the selenium on google collab and locating by xpath, but it can't seem to find the element
options = Options()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
driver = gs.Chrome(options=options)
driver.get('https://dropstab.com/coins/centrifuge/fundraising')
button = driver.find_element(by=By.XPATH, value='/html/body/div/div[1]/div/div[2]/main/div/article/div/div/section/div/div[1]/section[1]/div/div[1]/button')
button.click()
For reference, the button is each of the fundraising rounds if you scroll down (Series A, Venture Round, etc)
The code below will click on each of the Fundraising Rounds accordions and expand them. Before doing that, it closes the cookies banner.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
URL = "https://dropstab.com/coins/centrifuge/fundraising"
driver = webdriver.Chrome()
driver.maximize_window()
driver.get(URL)
time.sleep(2)
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#cookiesAgreementDescription + button"))).click()
rounds = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//h2[text()='Fundraising Rounds']//following-sibling::div/div")))
for _round in rounds:
_round.click()