javaamazon-web-servicesamazon-dynamodbamazon-kinesisamazon-kcl

Using KCL 2.x to get DynamoDBEvents from KinesisStream

From a KinesisClientRecord (record) that I obtained using KCL client, I’m able to get JSON object representing ddb update event from the stream by doing:

String recordData = StandardCharsets.UTF_8.decode(record.data()).toString();
JSONObject kinesisRecordObject = new JSONObject(recordData);

Now I want to access the fields that I got from the dynamoDB but I can’t deserialise it as its a DynamoDB JSON and not a standard one as explained in this post too Converting DynamoDB JSON to Standard JSON with Java. The solution given there works DynamoDBStreamRecord but not with KinesisClientRecord(which I’m working with) with Can you please tell me how this step is to be achieved? For KCL 1.x I found that KCL Adapter can be used, but I'm using KCL 2.x!

How can I deserialise it to get the fields in the DDB update?

Solution

  • When using KCL 2 you get CommittableRecord Object from Kinesis. From this you need to extract records().data() which is UTF 8 encoded(by default). The JSON String you get after decoding is then you can convert into desired POJO

    Something like this

    String recordData =
        new StringBuffer(StandardCharsets.UTF_8.decode(committableRecordObject.record().data()))
        .toString();