pythonmysqlamazon-web-servicesaws-rds-data-service

Python how to parse output of Amazon RDS Data Service


I'm struggling to parse output of ExecuteStatement action using Amazon RDS Data service https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html

Documentation is very poor and I can't find any meaningful examples.

I'm using Aurora MySql DB

Output is returned like this. This example is 2 rows of data returned by SELECT statement, I've replaced actual data with word "data"

 {'ResponseMetadata': {'RequestId': '955a6aee-5bad-4f87-a455-b83a10a8a31b', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '955a6aee-5bad-4f87-a455-b83a10a8a31b', 'content-type': 'application/json', 'content-length': '809', 'date': 'Tue, 02 Jun 2020 05:39:22 GMT'}, 'RetryAttempts': 0}, 'numberOfRecordsUpdated': 0, 'records': [[{'stringValue': 'data'}, {'longValue': data}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': ''}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}], [{'stringValue': 'data'}, {'longValue': data}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': ''}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}, {'stringValue': 'data'}]]}

I need to extract actually only this data fields, so I can use them for INSERT statement in another table.

As per documentation original response is dictionary data type, so I get 3 keys here 'ResponseMetadata', 'numberOfRecordsUpdated' and 'records'. As per documentation 'records' is Type: Array of arrays of Field objects and here is where I stuck. How I can extract Field objects from array of arrays inside dictionary?


Solution

  • There are many ways, but the most basic nested for-loop in python could suffice:

    data_values = []
    
    for record in result['records']:
        print()
        row_data = []
        for data_dict in record:
            #print(data_dict)
            for data_type, data_value in data_dict.items():
                print(data_type, data_value)
                row_data.append(data_value)
    
        data_values.append(row_data)
    
    print(data_values)  
    
    

    The code can for a good basis for modifications to suit your needs. It produces the following:

    stringValue data
    longValue data
    stringValue data
    stringValue data
    stringValue 
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    
    stringValue data
    longValue data
    stringValue data
    stringValue data
    stringValue 
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    stringValue data
    

    And data_values:

    [['data', 'data', 'data', 'data', '', 'data', 'data', 'data', 'data', 'data', 'data', 'data', 'data'], ['data', 'data', 'data', 'data', '', 'data', 'data', 'data', 'data', 'data', 'data', 'data', 'data']]