I have an issue with DRF, CursorPagination and Filters.
I have an endpoint. When I access the initial page of the enpoint I get a next URL
"next": "http://my-url/api/my-endpoint/?cursor=bz0yMDA%3D&date__gte=2025-04-25T10%3A00%3A00Z&date__lte=2025-04-26T10%3A00%3A00Z"
When I access this URL I get a previous URL
"next": "http://my-url/api/my-endpoint/?cursor=bz00MDA%3D&date__gte=2025-04-25T10%3A00%3A00Z&date__lte=2025-04-26T10%3A00%3A00Z",
"previous": "http://my-url/api/my-endpoint/?cursor=cj0xJnA9MjAyNS0wNC0yNSsxMCUzQTAwJTNBMDAlMkIwMCUzQTAw&date__gte=2025-04-25T10%3A00%3A00Z&date__lte=2025-04-26T10%3A00%3A00Z",
Now when I try to access the previous URL, I get an empty result list.
Here is the code for the endpoint
class RevenuePagination(CursorPagination):
page_size = 200
ordering = ['date', 'custom_channel_name', 'country_name', 'platform_type_code', 'id']
class RevenueFilter(django_filters.FilterSet):
class Meta:
model = Revenue
fields = {
'date': ['lte', 'gte'],
'custom_channel_name': ['exact'],
'country_name': ['exact'],
'platform_type_code': ['exact'],
}
class RevenueViewSet(viewsets.ModelViewSet):
permission_classes = [HasAPIKey]
queryset = Revenue.objects.all()
serializer_class = RevenueSerializer
filterset_class = RevenueFilter
pagination_class = RevenuePagination
@revenue_list_schema()
def list(self, request, *args, **kwargs):
return super().list(request, *args, **kwargs)
From what I get, the problem seems to be that the previous URL cursor (cj0xJnA9MjAyNS0wNC0yNSsxMCUzQTAwJTNBMDAlMkIwMCUzQTAw
) gets decoded to r=1&p=2025-04-25+10:00:00+00:00
, which is not right, because only the date field is in the cursor, while I have specidied 6 fields in the ordering of the pagination.
I tried it narrowing down the order to ['date', 'id'], but it does not work.
All the fields in the ordering are actual DB fields (DateTime, Char or Integer), no properties.
I am really struggling, tried debugging with Google AI and ChatGPT, both came to a dead end.
EDIT: I went in the DRF source code, and I found that the CursorPagination._get_position_from_instance
only takes into consideration the first field in the ordering list.
def _get_position_from_instance(self, instance, ordering):
field_name = ordering[0].lstrip('-')
if isinstance(instance, dict):
attr = instance[field_name]
else:
attr = getattr(instance, field_name)
return str(attr)
This is really strange, as per documentation the CursorPagination.ordering supports multiple fields. Or am I wrong?
EDIT2: I narrowed the problem down to having date value repeating, and filtering on the date value.
The filter is date__gte=2025-04-25 10:00:00
, and the first 10 pages have the same date value '2025-04-10 10:00:00'. The CursorPagination seems to be taking the first date value from the second page, and using that in the previous cursor. I think it needs something more unique. The combination fields provide uniqueness, but it does not work.
ordering
can (and often should!) be multiple fields.
But you're right that only the first field is stored in the cursor token. However, the other fields are still applied:
# rest_framework/pagination.py
class CursorPagination(BasePagination):
[...]
def paginate_queryset(self, queryset, request, view=None):
[...]
'''
The below part is where the ordering fields come into play.
They are not stored in the token, instead they are reapplied
on every hit of the paginator.
'''
# Cursor pagination always enforces an ordering.
if reverse:
queryset = queryset.order_by(*_reverse_ordering(self.ordering))
else:
queryset = queryset.order_by(*self.ordering)
So this part of the investigation is a red herring.
The lacking uniqueness across a large number of rows in the primary ordering fields is the primary part of this issue - if we exclude design decisions made by the DRF team.
You were on to the right idea at the end of your post. The cursor reversing logic is roughly as follows:
1. Take first element of current page
2. Generate a condition on the primary ordering field that excludes #1
3. Generate page with #2
For ordering fields that are unique (or are unique "enough" that multiple pages won't have the same ordering value), this is perfectly fine.
But for your dataset, the condition that rules out elements from the current page also rules out all the elements that are supposed to be on the previous page.
Initial page:
Next
page:
Previous
page:
SQL:
WHERE "event_event"."date" < '2025-04-25 08:00:00'
This is the condition that's built to rule out the 2nd page elements, but since the 1st page elements have that same value they get ruled out too.
The reason the subsequent ordering fields "don't matter" in this specific instance, is because that ordering doesn't take place until after the SQL has been generated, and since the query returns 0 rows, there's naturally nothing to order.
With your dataset, this isn't easily fixable with cursor pagination. You'd have to build cursor logic where the token is encoded with multiple fields instead of just the one, and that has its own complications. I don't know what all of them are, but I'm guessing there's either a limitation in the ORM that makes certain condition-combinations tricky to write, and/or an issue with how to generate/map multi-field orderings into proper SQL in a universally applicable way that can also be reliably reversed.
If you really want to hack it with cursor pagination, you could store the "current page" URL on the client's localstorage and use that URL for previous
instead of letting DRF reverse the token by itself.
But if you can't switch to a more unique primary ordering field, my best recommendation is switching to page number pagination.