htmlcssrweb-scrapingrvest

Hover pop-up text cannot be selected for rvest in R


I have been trying to scrape the rate data from this link: https://icoholder.com/en/patron-17242

You can see on the right side, the web plot is given, and if you stay on the names (i.e. ICO Profile) it give the score.

However, the selector or Xpath for that specific info cannot be extracted.

< span class="ico-radar-names ico-profile" data-name="profile" data-value="4.3" >ICO Profile< /span > 

You can see the tag of the score is data-value. But when I try it in R with rvest, the result I get for the tag is:

read_html('https://icoholder.com/en/patron-17242') %>% html_nodes('.ico-profile') [1]

< span class="ico-radar-names ico-profile" data-name="profile" >ICO Profile< /span >

You can see that when the node is recorded with rvest, the data value is lost.

Do you know why it happens and how I can fix it?

Cheers!


Solution

  • Why you can't use it is that information is not in page source. Your browser automatically sends more requests and it is received and rendered later.

    Use the hidden API instead: https://icoholder.com/en/get_rating/17242

    The result is returned in a simple JSON format, very easy to use:

    [{"category":"profile","value":4.3},{"category":"vision","value":4.8},{"category":"activity","value":4},{"category":"potential","value":4.5},{"category":"product","value":4.5},{"category":"team","value":4.8},{"general":4.49},{"on":true}]
    

    The way to find it is to open developer tools in Chrome. And it's under the network tab.

    enter image description here


    You can also use Rselenium, but it should be an overkill.