I am trying to scrape this site https://shopee.co.id/S-B-Golden-Curry-Bumbu-Kari-Japanese-Curry-Mix-Medium-Hot-220-gr-i.65323877.3343602079?sp_atk=0b471505-cfca-491b-b7a2-453d455eecf6&xptdk=0b471505-cfca-491b-b7a2-453d455eecf6
I am using python httpx and the below is my code
url=f"https://shopee.co.id/api/v4/item/get?shopid=65323877&itemid=3343602079"
headers = { "User-Agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
}
resp=httpx.get(url,headers=headers)
The returned json is as follows:
{'error': None,
'error_msg': None,
'data': {'itemid': 3343602079,
'shopid': 65323877,
'userid': 0,
'price_max_before_discount': -1,
'has_lowest_price_guarantee': False,
'price_before_discount': 0,
'price_min_before_discount': -1,
'exclusive_price_info': None,
'hidden_price_display': None,
'price_min': 9315023372,
'price_max': 9315023372,
'price': 9315023372, ...
I see that the price shown in the json is different than the one on the web;7490000000. Can someone please help?
Try to send a request without User-Agent header. It seems that the server sends different content based on this information:
import requests
url = 'https://shopee.co.id/api/v4/item/get?shopid=65323877&itemid=3343602079'
data = requests.get(url).json()
print(data)
Prints:
{
"error": None,
"error_msg": None,
"data": {
"itemid": 3343602079,
"shopid": 65323877,
"userid": 0,
"price_max_before_discount": -1,
"has_lowest_price_guarantee": False,
"price_before_discount": 0,
"price_min_before_discount": -1,
"exclusive_price_info": None,
"hidden_price_display": None,
"price_min": 7490000000,
"price_max": 7490000000,
"price": 7490000000,
"stock": 0,
...