So im trying to grab all the reviews from yelp for the hotel: https://www.yelp.com/biz/capri-laguna-laguna-beach
I have my code below, but I'm unable to pull all the reviews.. I am only able to pull one.. can someone please assist?
I would ideally love to pull all the yelp reviews for this establishment
import time
import random
from bs4 import BeautifulSoup as bs
import urllib.request as url
html = urllib.request.urlopen('https://www.yelp.com/biz/capri-laguna-laguna-beach').read().decode('utf-8')
soup = bs(html, 'html.parser')
relevant= soup.find_all('p', class_='comment__09f24__gu0rG css-qgunke')
for div in relevant:
for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
text = html_class.find('span')
review = html_class.getText(
print(review)
You are only getting one review printed because you wrote the print statement outside the loop.
relevant= soup.find_all('p', class_='comment__09f24__gu0rG css-qgunke')
for div in relevant:
for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
text = html_class.find('span')
review = html_class.getText()
print(review)
If you implement the code above, you'll get all 10 reviews printed.
To store all 10 reviews in a list do this,
reviews = []
for div in relevant:
for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
text = html_class.find('span')
review = html_class.getText()
reviews.append(review)