pythonbeautifulsoup

Python (BeautifulSoup) Only 1 result


I know there are similar questions to this one that are answered which I already tried applying and didn't fix my problem.

My problem is that on this website: http://books.toscrape.com/catalogue/page-1.html there are 20 prices and when I try to scrape the prices, I only get the first price but not other 19.

Here's the code

from bs4 import BeautifulSoup
import requests
URL = 'http://books.toscrape.com/catalogue/page-1.html'
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
results = soup.find_all("div", class_ = "col-sm-8 col-md-9")

for i in results :
    prices = i.find("p", class_ = "price_color")
    print(prices.text.strip())
    print()

Solution

  • this code should work!

    import requests
    from bs4 import BeautifulSoup
    
    
    URL = 'http://books.toscrape.com/catalogue/page-1.html'
    page = requests.get(URL)
    soup = BeautifulSoup(page.content, 'html.parser')
    
    list_of_books = soup.select(
        # using chrom selector
        '#default > div > div > div > div > section > div:nth-child(2) > ol > li'
    )
    
    for book in list_of_books:
        price = book.find('p', {'class': 'price_color'})
        print(price.text.strip())
    
    

    i just used chorme selector this is a screenshot of it

    you are using the find and find_all in the wrong places.