mysqlboto3serverlessamazon-auroraaws-rds-data-service

How do I paginate my mysql query using AWS rds-data using boto3 and AWS-aurora?


I am querying AWS Aurora (MySQL) serverless from AWS Lambda using Boto3. I want to query a table that has more than 10k records. My query can fetch records more than 7k. Now how can I paginate this data, on database level or Lambda level, so that I can get fewer data per page and send that small dataset to the user interface to display? I cannot send the entire dataset of 7k records.

Here is my Python code

rdsDataClient.execute_statement(resourceArn=cluster_arn,
                                secretArn=secret_arn,
                                database='myTestDB',
                                sql=sqlQuery,
                                parameters = paramSet
                               )

I am aware of the SQL query LIMIT and OFFSET but want to know if there is any other better way of doing this. Please help.

Thanks.


Solution

  • Yes I used the approach of passing limit and offset to sqlQuery. Here is what I did

    def lambda_handler(event, context):
        queryParameters = event["queryStringParameters"]
        search = getQueryStringElement("search", queryParameters, '')
        pageSize = int(getQueryStringElement("pageSize", queryParameters, 5))
        page = int(getQueryStringElement("page", queryParameters, 1))
        
        limit = pageSize
        offset = (pageSize * page) - pageSize
        
        sqlQuery = sqlQuery + ' LIMIT {} OFFSET {} ;'.format(limit, offset)
    
        rdsDataClient.execute_statement(resourceArn=cluster_arn,
                                    secretArn=secret_arn,
                                    database='myTestDB',
                                    sql=sqlQuery,
                                    parameters = paramSet
                                   )
    

    This solution worked perfect for my use case where I used to show the paginated list of records in UI

    Thanks