rssgoogle-reader

How does Google Reader get every item in an RSS feed?


Slashdot's RSS feed is http://rss.slashdot.org/Slashdot/slashdot. If I download the XML file directly, I only get a few of the posts from today. However, if I subscribe to the feed in Google Reader, and keep scrolling down in their "infinite scroll" interface, it seems like I can get an arbitrary number of Slashdot posts from the past - maybe I can get every Slashdot post ever?

  1. How does Google Reader retrieve an unlimited number of posts from an RSS feed?
  2. How can I do the same?

Solution

  • Google follows one instance of the feed for all its users, so they've been tracking and storing Slashdot articles, for example, long before any new subscriber starts reading.

    To do the same, you would have to poll the RSS feeds you want at regular intervals and store any unique articles you find locally.