Paging library: Jumping list items + always same data at the end

I am trying to implement an infinite list with the Paging library, MVVM and LiveData.

In my View (in my case my fragment) I ask for data from the ViewModel and observe the changes:

override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)

        viewModel.getItems("someSearchQuery")
        viewModel.pagedItems.observe(this, Observer<PagedList<Item>> {
            // ItemPagedRecyclerAdapter
            // EDIT --> Found this in the official Google example
            // Workaround for an issue where RecyclerView incorrectly uses the loading / spinner
            // item added to the end of the list as an anchor during initial load.
            val layoutManager = (recycler.layoutManager as LinearLayoutManager)
            val position = layoutManager.findFirstCompletelyVisibleItemPosition()
            if (position != RecyclerView.NO_POSITION) {
                recycler.scrollToPosition(position)
            }
        })
}

In the ViewModel I fetch my data like this:

private val queryLiveData = MutableLiveData<String>()

private val itemResult: LiveData<LiveData<PagedList<Item>>> = Transformations.map(queryLiveData) { query ->
    itemRepository.fetchItems(query)
}

val pagedItems: LiveData<PagedList<Item>> = Transformations.switchMap(itemResult) { it }

private fun getItems(queryString: String) {
    queryLiveData.postValue(queryString)
}

In the repository I fetch for the data with:

fun fetchItems(query: String): LiveData<PagedList<Item>> {
    val boundaryCallback = ItemBoundaryCallback(query, this.accessToken!!, remoteDataSource, localDataSource)

    val dataSourceFactory = localDataSource.fetch(query)

    return dataSourceFactory.toLiveData(
        pageSize = Constants.PAGE_SIZE_ITEM_FETCH,
        boundaryCallback = boundaryCallback)
}

As you might have already noticed, I used the Codelabs from Google as an example, but sadly I could not manage to make it work correctly.

class ItemBoundaryCallback(
    private val query: String,
    private val accessToken: AccessToken,
    private val remoteDataSource: ItemRemoteDataSource,
    private val localDataSource: Item LocalDataSource
) : PagedList.BoundaryCallback<Item>() {

    private val executor = Executors.newSingleThreadExecutor()
    private val helper = PagingRequestHelper(executor)

    // keep the last requested page. When the request is successful, increment the page number.
    private var lastRequestedPage = 0

    private fun requestAndSaveData(query: String, helperCallback: PagingRequestHelper.Request.Callback) {
        val searchData = SomeSearchData()
        remoteDataSource.fetch Items(searchData, accessToken, lastRequestedPage * Constants.PAGE_SIZE_ITEMS_FETCH, { items ->
            executor.execute {
                localDataSource.insert(items) {
                    lastRequestedPage++
                    helperCallback.recordSuccess()
                }
            }
        }, { error ->
            helperCallback.recordFailure(Throwable(error))
        })
    }

    @MainThread
    override fun onZeroItemsLoaded() {
        helper.runIfNotRunning(PagingRequestHelper.RequestType.INITIAL) {
            requestAndSaveData(query, it)
        }
    }

    @MainThread
    override fun onItemAtEndLoaded(itemAtEnd: Item) {
        helper.runIfNotRunning(PagingRequestHelper.RequestType.AFTER) {
            requestAndSaveData(query, it)
        }
    }

My adapter for the list data:

class ItemPagedRecyclerAdapter : PagedListAdapter<Item, RecyclerView.ViewHolder>(ITEM_COMPARATOR) {

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): RecyclerView.ViewHolder {
        return ItemViewHolder(parent)
    }

    override fun onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) {
        val item = getItem(position)
        if (item != null) {
            (holder as ItemViewHolder).bind(item, position)
        }
    }

    companion object {
        private val ITEM_COMPARATOR = object : DiffUtil.ItemCallback<Item>() {
            override fun areItemsTheSame(oldItem: Item, newItem: Item): Boolean =
                olItem.id == newItem.id

            override fun areContentsTheSame(oldItem: Item, newItem: Item): Boolean =
                oldItem == newItem
        }
    }
}

My problem right now is: The data is fetched and saved locally and is even displayed correctly in my list. But the data seems to be "looping", so there is always the same data showing despite there are different objects in the database (I checked with Stetho, about several hundred). Curiously the last item in the list is also always the same and sometimes there are items reloading while scrolling. Another problem is that it stops reloading at some point (sometimes 200, sometimes 300 data items).

I thought it might be because my ITEM_COMPARATOR was checking wrongly and returning the wrong boolean, so I set both to return true just to test, but this changed nothing.

I was also thinking of adding a config to the LivePagedListBuilder, but this also changed nothing. So I am a little bit stuck. I also looked into some examples doing it with a PageKeyedDataSource etc., but Google's example is working without it, so I want to know why my example is not working. https://codelabs.developers.google.com/codelabs/android-paging/index.html?index=..%2F..index#5

---

Edit:

Google does have another example in their blueprints. I added it to the code. https://github.com/android/architecture-components-samples/blob/master/PagingWithNetworkSample/app/src/main/java/com/android/example/paging/pagingwithnetwork/reddit/ui/RedditActivity.kt.

Now it is loading correctly, but when the loading happens, some items in the list still flip.

---

Edit 2:

I edited the BoundaryCallback, still not working (now provided the PagingRequestHelper suggested by Google).

---

Edit 3:

I tried it just with the remote part and it works perfectly. There seems to be a problem with Room/the datasource that room provides.

Ok, just to complete this issue, I found the solution.

To make this work, you must have a consistent list-order from your backend/api-data. The flipping was caused by the data that was constantly sent in another order than before and therefore made some items in the list "flip" around.

So you have to save an additional field to your data (an index-like field) to order your data accordingly. The fetch from the local database in the DAO is then done with a ORDER BY statement. I hope I can maybe help someone who forgot the same as I did:

@Query("SELECT * FROM items ORDER BY indexFromBackend")
abstract fun fetchItems(): DataSource.Factory<Int, Items>