pythonselenium

How do you accept cookies across multiple websites?


I have a list of domains that I would like to loop over and screenshot using selenium. However, the cookie consent column means the full page is not viewable. Most of them have different consent buttons - what is the best way of accepting these? Or is there another method that could achieve the same results?

urls for reference: docjournals.com, elcomercio.com, maxim.com, wattpad.com, history10.com

| URL            | cookie acceptance |
| -------------- | ----------------- |
| docjournals.com| AGREE             |
| elcomercio.com | Consentir         |
| maxim.com      | Consent           |
| wattpad.com    | I Accept          |
| history10.com  | AGREE             |

Solution

  • To get around the XPATH selectors varying from page to page you can use driver.current_url and use the url to figure out which selector you need to use.
    Or alternatively if you iterate over them anyways you can do it like this:

    page_1 = {
        'url' : 'docjournals.com'
        'selector' : 'example_selector_1'
    }
    
    page_2 = {
        'url' = 'elcomercio.com'
        'selector' : 'example_selector_2'
    }
    
    pages = [page_1, page_2]
    for page in pages:
        driver.get(page.url)
        driver.find_element(By.XPATH, page.selector).click()