I'm trying to extract a data from this website, somehow I get nothing out of any text I'm trying to get. I'm using Xidel to scrape the data.
xidel -e '//span[@class="main-price"]/text()' 'https://www.tokopedia.com/emas/harga-hari-ini'
**** Retrieving (GET): https://www.tokopedia.com/emas/harga-hari-ini ****
**** Processing: https://www.tokopedia.com/emas/harga-hari-ini/ ****
It should at least returning Rp or some numbers. But i'm not sure why it returning null. The other website i'm trying was just fine.
The target website is one of those sites in which the content is dynamically loaded using javascript. A simple way to confirm it is to go to the page, view it, then disable javascript in your browser and reload the page. In the case of this particular page, you'll see it's entirely blank.
There are a couple of ways to handle it, but unless I'm sorely mistaken, xidel isn't one of them. Start by taking a look at this.