I have an AWS SQS queue that is subscribed to an SNS topic. The queue currently has two consumers:
(Using terraform, Java)
The issue is that at any given time, only one of these consumers receives a message from the queue. I understand that SQS is designed to deliver each message to only one consumer. However, I want both consumers to process the same message independently.
Questions:
Code:
@SqsListener("${app.consumer.shopper-order.queue}")
public void listen(String message) throws JsonProcessingException {
// do stuff
}
Any help or guidance would be greatly appreciated!
The correct architecture would be:
This way, each message is sent to both queues and each consumer can independently consume the messages at their own pace. It is then easy to add additional consumers in future.
Alternatively, send messages to an Amazon Kinesis stream and have the consumers pull messages from the stream. Multiple consumers can read from the same stream, each with their own 'current pointer'. They can also replay messages rather than deleting them after consumption.
Consuming from an SQS queue is easier (and cheaper) than consuming from a Kinesis stream.