pythonseleniumweb-scrapingselenium-chromedriver

Take screenshot of full web page with Selenium Python with Chrome driver


I want to take a screenshot of the full page of a website. Below are the website conditions

  1. It requires authentication with a sign in form
  2. The website I want to screenshot is rendered by Javascript (dynamic rendering), not static rendering (inside the body tag is just script tags)

I did try some solutions I can find, but none of them work as I desired. Below are some attempt I tried

  1. Link This one required using headless chrome, which make me unable to fill the sign in form
  2. When I try to remove the headless options, I get this error "message":"Cannot take screenshot with 0 height."I guess this is because the content is rendered by Javascript. Note that I already give it some time to render with time.sleep(5)
  3. When I try to choose another element, the result is varying. Some gives same error message as above, some give me the screenshot but not full page, just the visible part (same result as using browser.save_screenshot()). Below is the code I tried.

def S(X): return browser.execute_script(
                'return document.querySelector("#main-layout-content").scroll'+X
)
browser = webdriver.Chrome()
browser.get('some link go here')
browser.set_window_size(S('Width') + 100, S('Height') + 1000)
#The print statement below work
print(S('Height')) 
browser.find_element_by_id('main-layout-content').screenshot('web_screenshot.png')

Is there any way to achieve what I need?


Solution

  • Here is what I did to get the full-size screenshot of a webpage

    from selenium import webdriver
    browser = webdriver.Chrome()
    
    # We go to the webpage here and wait 2 seconds for fully load
    browser.get('https://someURL')
    time.sleep(2)
    
    # Where to save the picture
    directory = '/home'
    # Video title
    title = 'Hello world'
    
    try:
            # We try to get the top-level component in which all out page is its children
            # This is different for each website
            elem = browser.find_element_by_id('main-layout-content')
            # Get the height of the element, and adding some height just to be safe
            total_height = elem.size['height'] + 1000
            # Set the window size - what is the size of our screenshot
            # The width is hardcoded because the screensize is fixed for each computer
            browser.set_window_size(1920, total_height)
            # Wait for 2 seconds
            time.sleep(2)
            # Take the screenshot
            browser.find_element_by_id(
                'main-layout-content').screenshot(f'./{directory}/{title}.png')
    except SystemError as err:
            print('Take screenshot error at' + title)