pythonselenium-webdriverweb-scraping

Cannot get data from dropdown using Selenium

I've tried all logical divs to get the dropdown list, but could not make it '.click'. The URL I am trying to get the information from is https://krisha.kz/. I don't get any mistakes but when the code is running Chrome opens the site and just closes without clicking the link to make dropdown visible, so I could scrape data from it.

enter image description here

from selenium import webdriver
from selenium.webdriver.common.by import By
import csv

driver = webdriver.Chrome()
url = f'https://krisha.kz'
driver.get(url)

div = driver.find_element(By.ID, value='region-selected-value')
div.click()
#selector = div.find_element(By.TAG_NAME,value='select')

print(div.text)
Solution

  • As I have mentioned in the comment, your code to click on that dropdown element is just fine. I have tested it and it clicks the dropdown element opening the dropdown options. You just need to put a time.sleep(20) at the end so that you see it before the browser closes down.

    Having said that, as I understand you are looking to scrape all the dropdown values. If that is correct, see the code below:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome()
url = f'https://krisha.kz'
driver.get(url)
select = Select(driver.find_element(By.NAME, "das[live.rooms]"))
options = select.options
for option in options:
    print(option.text)

    Result:

    1 - комн.
1-2 - комн.
2 - комн.
2-3 - комн.
3 - комн.
3-4 - комн.
4 - комн.
4-5 - комн.
5 и более комн.

Process finished with exit code 0