We have a use case where messages are consumed from RabbitMQ, transform it and push the record to the Kinesis stream. Each message in RabbitMQ corresponds to a record in the Kinesis stream. AmazonKinesisAsync supports PutRecordsRequestEntry where multiple records can be grouped to make a single HTTP call.
My question is there any out-of-box component in Spring Integration that helps to group the messages from rabbitmq into a batch based on the configurable amount of records size before pushing to Kinesis?
See an Aggregator EI pattern and its implementation in Spring Integration: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#aggregator. So, before pushing the message to the KinesisMessageHandler
, you have an AggregatingMessageHandler
for batching those message from RabbitMQ by some configured size and emitting a PutRecordsRequest
container object.