postgresqlamazon-rdslogical-replicationaws-dms

Can I configure AWS RDS to only stream INSERT operations to AWS DMS?


My requirement is to stream only INSERTs on a specific table in my db to a Kinesis data stream.

I have configured this pipeline in my AWS environment:

RDS Postgres 13 -> DMS (Database Migration Service) -> KDS (Kinesis Data Stream)

This setup works correctly but it processes all changes, even UPDATEs and DELETEs, on my source table.

What I've tried:

Of course I could simply ignore records originated from non-INSERT operations in my Kinesis consumer, but this doesn't look like a cost-efficient implementation.

Is there any way to meet my requirements using these AWS services (RDS -> DMS -> Kinesis)?


Solution

  • Well DMS does not have this capability .

    If you want only INSERT to be send to Kinesis in that case you can have a lambda function on every INSERT of RDS . Lambda function can be configured as trigger for INSERT .

    You can invoke lambda only for INSERT and write to Kinesis directly .

    Cost wise also this will be less . In DMS you are paying for Replication instance even when not in use .

    For detailed reference Stream changes from Amazon RDS for PostgreSQL using Amazon Kinesis Data Streams and AWS Lambda