This is my first post on Stack Overflow, so apologies in advance if anything is not standard practice.
I am trying to download two csv files from a website in Python using Selenium and ChromeDriver. The two csv files that I want to download are under the Index Holdings and Portfolio Holdings tabs. To download the csv files, the CSV FILE buttons can be clicked under each of the aforementioned tabs.
Below is an MRE of my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions as EC
def press_button(wait, driver, xpath):
button = wait.until(EC.element_to_be_clickable((By.XPATH, xpath)))
ActionChains(driver).move_to_element(button).click().perform()
url = "https://www.sectorspdrs.com/mainfund/XLC"
csv_xpath = "//span[contains(text(), 'Download a Spreadsheet')]/following-sibling::button[contains(text(), 'CSV File')]"
tab_xpath = "//a[contains(text(), 'Portfolio Holdings')]"
service = Service(executable_path="chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get(url)
wait = WebDriverWait(driver, 10)
press_button(wait, driver, csv_xpath) # Download the csv file under Index Holdings
press_button(wait, driver, tab_xpath) # Press the Portfolio Holdings tab
press_button(
wait, driver, csv_xpath
) # Download the csv file under Portfolio Holdings (error occurs here)
Below is the error that I encounter.
selenium.common.exceptions.TimeoutException: Message:
button = wait.until(EC.element_to_be_clickable((By.XPATH, xpath)))
Below are my current versions
I assume I can use the same xpath for downloading both csv files, but I might be mistaken.
if anyone could help me resolve this issue, that would be greatly appreciated. Let me know if I should add any more detail.
There are actually two different div elements with same structure at the same position on the page, you can get them by using xpath //div[@class="tab-content"]/div[@role="tabpanel"], which will return you 4 elements, and the first two are them. Let call them index_tab and portfolio_tab.
When you click the Index Holdings or Portfolio Holdings tabs, index_tab and portfolio_tab are alternately displayed. One is shown, while the other is set to display:none.
So, you must download two files with xpath with their own buttons.
Your xpath //span[contains(text(), 'Download a Spreadsheet')]/following-sibling::button[contains(text(), 'CSV File')] should return 3 elements. The first two are what you need to click.