pythonseleniumweb-scrapingdirectorycreate-directory

Saving pictures to a new folder each iteration in Python


I practice my web scraping skills in Python and Selenium library. My goal is to download images from a real estate website (comparis.ch). I have a list of links where each link is an apartment. I want to save photos for each apartment in a new folder for each link (like apartment1, apartment2...). Can't figure out how to do that, maybe someone can help, I am quite new to Python. Thank you ;)

for link in links:
    url = link
    driver.get(url)

    # scraping pictures from the website
    # finding number of grey circles that indicate photos
    circles = len(driver.find_elements_by_class_name("svg-inline--fa.fa-circle.fa-w-16.css- 
    1xkwzfp"))+1
    print("{} photos found at the website.".format(circles))

    # creating a set, since duplicates are excluded in the set
    images_urls = set()
    for n in range(circles):
        # finding image containers
        images_containers = driver.find_element_by_class_name("css- 
        ze3zoq").find_elements_by_tag_name("img")

    for image in images_containers:
        # scraping urls from containers and store it in a set to avoid duplicates
        images_urls.update([image.get_attribute("src")])
        # click to scroll photos to the right and thus upload more photos
        driver.find_element_by_class_name("css-11m3oda.excbu0j2").click()

    # download photos (HOW TO SAVE THEM TO A NEW FOLDER EACH TIME?)
    for i in images_urls:
        # Download the images using requests library
        with open("C:/Users/potek/Jupyter_projects/apartments/{}".format("Comparis"+str(time.time())+".jpg"), "wb") as f: # comment 
        f.write(requests.get(i).content)```


Solution

  • Edit line 1 to:

    for num, link in enumerate(links):
    

    Append this on line 26 (after for i in images_urls):

    path = "C:/Users/potek/Jupyter_projects/apartments{}".format(num)
    os.makedirs(path) # don't forgate to import os
    

    Edit with open("C:/Users/potek/Jupyter_projects/apartments/{}".format("Comparis"+str(time.time())+".jpg"), "wb") as f::

    with open(os.path.join(path, "Comparis"+str(time.time())+".jpg"), "wb") as f: