I am trying to scrape this page https://www.anesishome.gr/%CE%B2%CF%81%CE%B5%CF%86%CE%B9%CE%BA%CE%AC-159#!/ I need the name and price of each product for the first 5 pages. The problem is tha my code gives the results of the first page 5 times. As if I dont't change the url for the next pages. What am I doing wrong? Thank you!
from urllib.request import urlopen
from bs4 import BeautifulSoup
for i in range(5):
page="https://www.anesishome.gr/%CE%B2%CF%81%CE%B5%CF%86%CE%B9%CE%BA%CE%AC-159#!/page-{}".format(i)
html = urlopen(page)
soup=BeautifulSoup(html, "html.parser")
pin=[None]*240
puk=[None]*240
k=soup.find("ul", class_="product-grid row")
titles=k.find_all("a", class_="product_image")
i=0
for title in titles:
pin[i]=title.get("title")
i=i+1
t=soup.find("ul", class_="product-grid row")
prices=t.find_all("span", class_="price")
i=0
for price in prices:
puk[i]=price.get_text()
i=i+1
x=0
with open('vrefika.txt', 'w') as f:
for x in range(0,i):
print(pin[x])
print("price=",puk[x])
string=pin[x]
f.write(string+"\n")
string=puk[x]
f.write(string+"\n")
I made a demo for you, hope this will help:
import requests
from bs4 import BeautifulSoup
for i in range(1, 6): # page start at 1, end with 5
page="https://www.anesishome.gr/%CE%B2%CF%81%CE%B5%CF%86%CE%B9%CE%BA%CE%AC-159?p={}".format(i)
html = requests.get(page)
soup = BeautifulSoup(html.text, 'lxml')
product_list = soup.find('div', id="product_list") # get product_list
for item in product_list('li', class_='ajax_block_product'): # itertate over each product
title = item.find('h5').text
price = item.find(class_="price").text
print(title, price)
out:
ΠΕΤΣΕΤΑ ΒΡΕΦΙΚΗ NEF - NEF HAPPY DAY MINT 40X60 2,66 €
ΠΑΙΔΙΚΗ ΠΕΤΣΕΤΑ ΧΕΡΙΩΝ NEF - NEF SNOOPY FRIENDS 3,50 €
Πετσετάκια ώμου σε σετ 3 τεμαχίων Pierre Cardin 4,80 €
Σαλιάρα Pierre Cardin 74 5,60 €
Σαλιάρα Pierre Cardin 135 5,60 €
ΒΡΕΦΙΚΟ ΜΑΞΙΛΑΡΙ ΜΑΛΑΚΟ NEF-NEF BALLFIBER 6,75 €
ΣΑΛΙΑΡΑ NEF-NEF TINY FRIENDS 7,20 €
ΣΑΛΙΑΡΑ NEF-NEF PLAY NOW 7,20 €
Pierre Cardin baby design 035 Πάνες Φανελένιες 7,20 €
Pierre Cardin baby design 036 Πάνες Φανελένιες 7,20 €
Baby Oliver design 212 Πάνες χασέ 7,20 €