javascriptpythonselenium-webdriverweb-scraping

How to click a pagination page number in python using Selenium?

I've been struggling to figure out how to click on a page number of a pagination class for a couple of days now. I've tried a bunch of methods but I can only get as far as highlighting the container of the number, not actually clicking it. What it looks like when I click on the page number

This is the source code for the pagination in the website:

<nav aria-label="pagination">
    <ul class="pagination">
        <li class="page-item false" data-page-number="<-">
            <a class="page-link" id="page-link«" href="#">«</a>
        </li>
        <li class="page-item active" data-page-number="1">
            <a class="page-link" id="page-link1" href="#">1</a>
        </li>
        <li class="page-item " data-page-number="2">
            <a class="page-link" id="page-link2" href="#">2</a>
        </li>
        <li class="page-item " data-page-number="3">
            <a class="page-link" id="page-link3" href="#">3</a>
        </li>
        <li class="page-item " data-page-number="4">
            <a class="page-link" id="page-link4" href="#">4</a>
        </li>
        <li class="page-item " data-page-number="5">
            <a class="page-link" id="page-link5" href="#">5</a>
        </li>
        <li class="page-item false" data-page-number="->">
            <a class="page-link" id="page-link»" href="#">»</a>
        </li>
    </ul>
</nav>

This is my current code that I use to locate and click the page numbers:

try:
    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "ul.pagination")))
    next_page_number = page_number + 1
    driver.find_element(By.XPATH, f'//*[@id="page-link{next_page_number}"]').click()
    time.sleep(2)

except NoSuchElementException or TimeoutException:
    break

I've also tried to replace the .click() with a different method of executing using driver.execute_script("arguments[0].click();", page), but it didn't work any better.

driver.find_element(By.XPATH, f"//a[@href='#'][text()='{next_page_number}']").click() - no different.

driver.find_element(By.CSS_SELECTOR, "ul.pagination").find_element(By.LINK_TEXT, f"{next_page_number}").click() - ;-;

I've tried finding the number using css selector as well but that didn't help either. I've also tried instead of click(), using send_keys(Keys.ENTER).

Overall I'm very lost, so any help or guidance would be greatly appreciated.

EDIT: The specific website I'm looking at is https://www.xior-booking.com

Solution

  • I have updated the nextpage locator you want to click as below to target with link element a

    //li[@data-page-number="{next_page_number}"]//a

    Then using a javscript click instead to click on the link

    driver.execute_script("arguments[0].click();", nextPageLink)

    Full Code

        from selenium.webdriver.support import expected_conditions as EC
    import undetected_chromedriver as uc

    driver = uc.Chrome()
    driver.get('https://www.xior-booking.com/#')

try:
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul.pagination")))
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '//li[@class="page-item '
                                                                                'active"]//a')))
    # first we get the current Active page number
    currentPageNumber = driver.find_element(By.XPATH, '//li[@class="page-item active"]//a').text
    # Then we calculate the next page number
    next_page_number = int(currentPageNumber) + 1
    nextPageLink = driver.find_element(By.XPATH, f'//li[@data-page-number="{next_page_number}"]//a')

    # doing a javascript click instead fo element.click
    driver.execute_script("arguments[0].click();", nextPageLink)
    # Increased wait here to visually verify
    time.sleep(20)

except NoSuchElementException or TimeoutException:
    print("exception Occurred")

    Clicked on page number 2 link enter image description here