I'm a complete beginner but have a question about my code for web scraping products on aliexpress.
The problem is that I only get 1 result instead of all of them.
from bs4 import BeautifulSoup
import requests
html_text = requests.get('https://dutch.alibaba.com/products/uhf_rfid_label.html?IndexArea=product_en&page=1').text
soup = BeautifulSoup(html_text, 'lxml')
producten = soup.find_all('div', class_ ='organic-list app-organic-search__list')
for product in producten:
product_naam = product.find('p', class_ = 'elements-title-normal__content large').text
jaren_actief = product.find('span', class_ = 'seller-tag__year flex-no-shrink').text
print(f'''Product naam: {product_naam} ''')
How do I get the information of ALL the products?
Will assume the information you want to get is the title of the product and the price as @Andrej_Kesley suggested in his answer.
Parsing the page as HTML, you can get only 8 products with Beautiful Soup as follows:
from bs4 import BeautifulSoup
import requests
url = 'https://dutch.alibaba.com/products/uhf_rfid_label.html?IndexArea=product_en&page=1'
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
products = soup.find_all('a', class_='elements-title-normal')
prices = soup.find_all('span', class_='elements-offer-price-normal__price')
num_products = len(products)
for i in range(num_products):
print("{:<20} {}".format(prices[i].text, products[i].text))
Output:
US$ 0,02-US$ 0,10 Aangepaste Tag Sticker Labels Lange Bereik Goedkope Passieve Papier Roll Uhf Rfid Label
US$ 0,08-US$ 0,12 Magazijn & Asset & Productie Lijnmanagement Lange Afstand Alien H3 Chip Uhf Rfid Papier Label
US$ 0,78-US$ 0,85 Hopeland hot selling UHF RFID Animal Ear Tag mini uhf rfid ear tag 860 960MHz 5m reading range rfid label tag uhf
US$ 0,03-US$ 0,06 Factory Outlet Uhf Rfid Sticker/Label Met Chip
US$ 0,07-US$ 0,12 Long Range Goedkope Passieve Papier Roll Uhf Rfid Chip Label Tag Sticker
US$ 0,02-US$ 0,04 Groothandel Asset Tracking R6 Chip Uhf Papier Label Rfid Uhf Inventaris Labels Uhf Sticker
US$ 0,04-US$ 0,13 Aangepaste Tags Alien H3 9662 H9 9640/M4E Chip Long Range Passieve Uhf Rfid Tag/ Label/ Sticker
US$ 0,08-US$ 0,20 Full Color Afdrukken Hf/Uhf Passieve Papier Roll Smart Nfc Rfid Label/Sticker/Tag