I am trying to extract Data from Quora for analysis purposes using Python, BeautifulSoup and Selenium. But I am unable to extract the JavaScript elements on the page. How should I extract them?
Here I am only trying to extract bio of the Quora Profile but I am not getting the text which comes after clicking the 'more' button.
~https://i.sstatic.net/20q72.jpg
# Extracting Bio
driver.find_element_by_class_name('ui_qtext_more_link').send_keys(Keys.ENTER)
bio = driver.find_element_by_class_name("ui_qtext_rendered_qtext").text
Please use the below line of code to first click on the "more" button and then fetch the expanded text of the profile.
import time
//Fetch the more button element first
WebElement moreButton = driver.find_element_by_xpath("(//a[@class='ui_qtext_more_link'])[1]");
//Click on the more button
moreButton.click();
time.sleep(3)
//Fetch the profileInfo element
WebElement profileInfo = driver.find_element_by_xpath("(//div[contains(@id,'expanded_content')]//span[@class='ui_qtext_rendered_qtext'])[1]");
//Store the bio in a string and use it further
String profileInfoBio = profileInfo.text;