I'm working on a project where I'd look to analyze the hygiene ratings of the restaurants listed on Zomato in Delhi. I was able to get the restaurant details using the Zomato /search API but the API does not provide the hygiene rating of the restaurants.
I tried scraping but I keep getting an error.
Web scraping code:
# import the library we use to open URLs
import urllib.request
headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'}
response = requests.get("https://www.zomato.com/ncr/pearl-boutique-bakery-cafe-greater-kailash-gk-2-new-delhi",headers=headers)
content = response.content
# open the url using urllib.request and put the HTML into the page variable
#page = urllib.request.urlopen(url)
# parse the HTML from our URL into the BeautifulSoup parse tree format
soup = BeautifulSoup(content, "lxml")
print(soup.prettify())
I keep getting the below error:
ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')",))
Is there any other way to extract the hygiene ratings of the restaurants from Zomato?
Specify http header User-Agent
to obtain the page.
For example:
import requests
from bs4 import BeautifulSoup
url = 'https://www.zomato.com/ncr/pearl-boutique-bakery-cafe-greater-kailash-gk-2-new-delhi'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0'}
soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')
print(soup.select_one('p:contains("HYGIENE RATING") + p').get_text(strip=True))
Prints:
5 - Excellent