apiweb-scrapingbeautifulsoupzomato-apihygiene

How can I extract hygiene ratings from Zomato?


I'm working on a project where I'd look to analyze the hygiene ratings of the restaurants listed on Zomato in Delhi. I was able to get the restaurant details using the Zomato /search API but the API does not provide the hygiene rating of the restaurants.

I tried scraping but I keep getting an error.

Web scraping code:

# import the library we use to open URLs
import urllib.request
headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'}
response = requests.get("https://www.zomato.com/ncr/pearl-boutique-bakery-cafe-greater-kailash-gk-2-new-delhi",headers=headers)

content = response.content
# open the url using urllib.request and put the HTML into the page variable
#page = urllib.request.urlopen(url)
# parse the HTML from our URL into the BeautifulSoup parse tree format
soup = BeautifulSoup(content, "lxml")
print(soup.prettify())

I keep getting the below error:

ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')",))

Is there any other way to extract the hygiene ratings of the restaurants from Zomato?

enter image description here


Solution

  • Specify http header User-Agent to obtain the page.

    For example:

    import requests
    from bs4 import BeautifulSoup
    
    url = 'https://www.zomato.com/ncr/pearl-boutique-bakery-cafe-greater-kailash-gk-2-new-delhi'
    headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0'}
    
    soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')
    
    print(soup.select_one('p:contains("HYGIENE RATING") + p').get_text(strip=True))
    

    Prints:

    5 - Excellent