Due to the limit rate of twitter API I would like to know how could I scrape the number of followers from a Twitter account with Node.js in a very very fast way? Do you think Node.js is the best solution to do it?
I have no idea of how it works but I thought that perhaps scraping to the Twitter mobile version ( example https://mobile.twitter.com/android) would be faster than the computer version.
What do you think? How could I do it?
Thanks!!
Now that the Twitter v1.0 API has been deprecated, you must use oauth signed requests. Which means Twitter knows when you abuse the system.
Docs on oath signed requests: https://dev.twitter.com/docs/auth/oauth
Docs on rate limiting: https://dev.twitter.com/docs/rate-limiting/1.1
Rate limiting applies to the API, not unauthenticated http requests which simply scrape the content.
If you just want the follower count and not the individual follower ids / details, you can simply scrape the page using request and use jQuery to filter down the results using selectors.
eg. https://twitter.com/foofighters
You can run this client side to preview the results:
$('.js-mini-profile-stats a.js-nav[data-nav=followers] strong').attr('title')