I am currently using the Motor client (v3.1.1) in a FastAPI application. When attempting to implement pagination, the following code works fine:
client = motor.motor_asyncio.AsyncIOMotorClient('mongodb://<my-connection-string>/')
database = client.test_database
collection = database.test_collection
pipeline = [
{ "$facet":
{"items":
[{ "$match": { }},
{"$sort": {"_id": -1}},
{ "$skip": pageskip },
{ "$limit": pagesize },
],
"totalCount": [{ "$count": "count" }]}}
]
cursor = collection.aggregate(pipeline)
However after a certain number of pages I get the following error:
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Sort exceeded memory limit of 104857600 bytes, but
did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt
in., full error: {'ok': 0.0, 'errmsg': 'Sort exceeded memory limit of 104857600
bytes, but did not opt in to external sorting. Aborting operation. Pass
allowDiskUse:true to opt in.', 'code': 292,
'codeName':'QueryExceededMemoryLimitNoDiskUseAllowed'}
I tried using [allowDiskUse][1] directly in aggregation but then I get the following:
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Unrecognized pipeline stage name: 'allowDiskUse',
full error: {'ok': 0.0, 'errmsg': "Unrecognized pipeline stage name: 'allowDiskUse'",
'code': 40324, 'codeName': 'Location40324'}
Thanks in advance if anyone knows how to implement allowDiskUse with the Motor client ...
You add it as flag to your aggregate
method:
cursor = collection.aggregate(pipeline, allowDiskUse=True)