pythonweb-scrapingbeautifulsouphtml-parsing

Pulling reviews from yelp - beautifulsoup


So im trying to grab all the reviews from yelp for the hotel: https://www.yelp.com/biz/capri-laguna-laguna-beach

I have my code below, but I'm unable to pull all the reviews.. I am only able to pull one.. can someone please assist?

I would ideally love to pull all the yelp reviews for this establishment

import time
import random
from bs4 import BeautifulSoup as bs

import urllib.request as url


html = urllib.request.urlopen('https://www.yelp.com/biz/capri-laguna-laguna-beach').read().decode('utf-8')

soup = bs(html, 'html.parser')

relevant= soup.find_all('p', class_='comment__09f24__gu0rG css-qgunke')

for div in relevant:
        for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
            text = html_class.find('span')
            review = html_class.getText(
            
print(review)


Solution

  • You are only getting one review printed because you wrote the print statement outside the loop.

    relevant= soup.find_all('p', class_='comment__09f24__gu0rG css-qgunke')
    
    for div in relevant:
      for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
        text = html_class.find('span')
        review = html_class.getText()          
        print(review)
    

    If you implement the code above, you'll get all 10 reviews printed.


    To store all 10 reviews in a list do this,

    reviews = []
    
    for div in relevant:
      for html_class in div.find_all('span',class_="raw__09f24__T4Ezm"):
        text = html_class.find('span')
        review = html_class.getText()
        reviews.append(review)