I am querying AWS Aurora (MySQL) serverless from AWS Lambda using Boto3. I want to query a table that has more than 10k records. My query can fetch records more than 7k. Now how can I paginate this data, on database level or Lambda level, so that I can get fewer data per page and send that small dataset to the user interface to display? I cannot send the entire dataset of 7k records.
Here is my Python code
rdsDataClient.execute_statement(resourceArn=cluster_arn,
secretArn=secret_arn,
database='myTestDB',
sql=sqlQuery,
parameters = paramSet
)
I am aware of the SQL query LIMIT and OFFSET but want to know if there is any other better way of doing this. Please help.
Thanks.
Yes I used the approach of passing limit and offset to sqlQuery. Here is what I did
def lambda_handler(event, context):
queryParameters = event["queryStringParameters"]
search = getQueryStringElement("search", queryParameters, '')
pageSize = int(getQueryStringElement("pageSize", queryParameters, 5))
page = int(getQueryStringElement("page", queryParameters, 1))
limit = pageSize
offset = (pageSize * page) - pageSize
sqlQuery = sqlQuery + ' LIMIT {} OFFSET {} ;'.format(limit, offset)
rdsDataClient.execute_statement(resourceArn=cluster_arn,
secretArn=secret_arn,
database='myTestDB',
sql=sqlQuery,
parameters = paramSet
)
This solution worked perfect for my use case where I used to show the paginated list of records in UI
Thanks