pythonbeautifulsoupurlretrieve

urlretrieve for image returns HTTP Error 403: Forbidden


Hei guys, I am trying to get an image using BeautifulSoup but I am getting an error when doing so:

Here is my code:

imgUrl = "https://www.residentadvisor.net/images/events/flyer/2017/7/no-0713-986042-front.jpg"
try:
    urlretrieve(imgUrl, "testPhytonImg.jpg")
except FileNotFoundError as err:
    print("something wrong with local path")
    print(err)   # something wrong with local path
except HTTPError as err:
    print("something wrong with url")
    print(err)  # something wrong with url

And this is the error I get: HTTP Error 403: Forbidden

And reason why I get this? Does the access to the image get's blocked because of something I do or is there another way for this?


Solution

  • This works for me. You need to add a request header

    import urllib.request
    url_address = "https://www.residentadvisor.net/images/events/flyer/2017/7/no-0713-986042-front.jpg"
    headers={'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11',
       'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
       'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
       'Accept-Encoding': 'none',
       'Accept-Language': 'en-US,en;q=0.8',
       'Connection': 'keep-alive'}
    request_=urllib.request.Request(url_address,None,headers) #The assembled request
    response = urllib.request.urlopen(request_)# store the response
    #create a new file and write the image
    f = open('00000001.jpg','wb')
    f.write(response.read())
    f.close()