[SOLVED] Pagination of Stargate document API for Cassandra

Pagination of Stargate document API for Cassandra

The Stargate Document API for Cassandra provides page-size to return all documents limited by a page size. The page-size is has max value of 20. How do I get the next documents; i.e 21 to 30?

Reading Documents using Stargate API for Cassandra

Solution

So when you run your query for the initial 20, you'll get a property in the returned JSON called pageState. Here's an example with page-size=5:

% curl --request GET \
  --url https://$ASTRA_DB_ID-$ASTRA_DB_REGION.apps.astra.datastax.com/api/rest/v2/namespaces/$ASTRA_DB_KEYSPACE/collections/hello_docs\?page-size\=5 \
  -H "X-Cassandra-Token: $ASTRA_DB_APPLICATION_TOKEN" \
  -H 'Content-Type: application/json'
{"pageState":"JGNlYjczMDc5LTE1NTItNGQyNS1hM2ExLWE2MzgxNWVlYTAyMADwf_____B_____","data":{
"58b12778-0efd-466e-89bf-66a4c99adee1":{"author":"Tom Clancy","title":"Red Rabbit"},
"77fe8690-f8d4-43b8-b1c9-a328318d4eae":{"author":"Tom Clancy","title":"Every Man a Tiger"},
"3177f86c-a633-4302-92a5-de4c15ab5840":{"author":"Tom Clancy","title":"Rainbow Six"},
"cefc7117-fc73-47b8-a965-099cf3a59269":{"author":"Tom Clancy","title":"Clear and Present Danger"},
"ceb73079-1552-4d25-a3a1-a63815eea020":{"author":"Tom Clancy","title":"The Cardinal of the Kremlin"}}}%

When I want to get the next 5 documents, I re-run the same query but add the page-state parameter with the value returned from the previous results:

&page-state\=JGNlYjczMDc5LTE1NTItNGQyNS1hM2ExLWE2MzgxNWVlYTAyMADwf_____B_____

When I add that to the original query, it looks like this:

% curl --request GET \
--url https://$ASTRA_DB_ID-$ASTRA_DB_REGION.apps.astra.datastax.com/api/rest/v2/namespaces/$ASTRA_DB_KEYSPACE/collections/hello_docs\?page-size\=5\&page-state\=JGNlYjczMDc5LTE1NTItNGQyNS1hM2ExLWE2MzgxNWVlYTAyMADwf_____B_____ \
-H "X-Cassandra-Token: $ASTRA_DB_APPLICATION_TOKEN" \
-H 'Content-Type: application/json'

 {"pageState":"JDUxMzVmMzFiLWY4NDYtNGZiNy1iNmFmLTk2YmU2OTMyNzlmMQDwf_____B_____","data":{
"f42cc7d8-81ba-41d5-b5c9-c13f8d0009c6":{"author":"Tom Clancy","title":"Patriot Games"},
"46df315f-7208-4859-af1c-96c4953f70c7":{"author":"Tom Clancy","title":"Without Remorse"},
"b23af545-4fdc-4ac5-907a-7132d7429d26":{"author":"Tom Clancy","title":"The Hunt for Red October"},
"c6d55ecd-1631-4d5c-8320-d111662f8837":{"other":"This is nonsensical stuff.","title":"Some Stuff"},
"5135f31b-f846-4fb7-b6af-96be693279f1":{"author":"Tom Clancy","title":"The Sum of all Fears"}}}%

tl;dr;

Add the pageState returned in the results, and you should get the next 20.

Edit 20211011

The Stargate documentation has been updated with detail to address this process.

Retrieving all documents with Paging