design-patternsbackendsystem-design

The best way to design an infinite scroll list on backend


I'm trying to design a page that supports infinite scroll. There is 1 important feature that we definitely want this to support: we want to be able to update the already loaded items if an item is changed/deleted.

For example, imagine your Twitter homepage is loaded and one of the loaded tweets gets deleted. Our ideal list needs to be able to get updated with the changes.

The normal way I had in mind was to call a method such as loadTweets(pageNumber = 1) and as the user scrolls down, pageNumber increments and we make the API call again. But this will not support keeping track of already loaded items.


Solution

  • The first option that springs to my mind is to model this as a stream of immutable events, e.g.:

    (Most recent events on top)

    You can implement this using a pub/sub mechanism, or emulating an event stream as a feed of events, like, for example, RSS or ATOM. Allamaraju Subrahmanyam's RESTful Web Services Cookbook has some good content on the latter option. For the first option, you may want to consult Enterprise Integration Patterns by Hohpe and Woolf.

    It'll be the responsibility of the UI to interpret the events by, for example, removing or updating items as new events appear.