I have a fairly simple scenario. I want to implement a Feature Flags service. The plan is to use DynamoDB to store the feature flags using a composite key. The table will be partitioned by the application name (since I want to isolate each applications feature flags from one another), and sorted by the feature name. Each feature flag has an enabled flag, a description, and optionally an options hash that contains additional configuration information.
Now, I want my various applications / services to dynamically respond to changes to these feature flags. I don't want each application to have to manually pull the changes down every so often - I want to push the changes down to each application. This is especially true for our front-end react applications.
My initial approach to this is to enable the DynamoDB stream on the feature-flags table and then use an Event Bridge Pipe from the table to an SNS Topic. Backend services can then register their own SQS Queue to this if so desired. However, for my initial setup there will be just one SQS Queue used by the feature-flags service itself.
I'm using rails for the feature-flags service, and I will use Action Cable to support web sockets. The feature-flags service will consume the events in the SQS Queue - pushing the notifications via web sockets to any subscribers.
I've been testing things locally, and it looks like this will all flow nicely. The only thing that I am concerned about is how to avoid out-of-order event processing with respects to individual items. DynamoDB Streams guarantees that events pertaining to an individual item / record will appear in the stream in-order. However, the Event Bridge Pipe does not appear to support connecting a DynamoDB Stream (source) to a FIFO SNS Topic (target). This forces me to use a standard SNS Topic and a standard SQS Queue (subscriber), neither of which provide any guarantee about the order that messages will be processed...
So, my question is how to adjust my configuration and use of AWS services so that my feature-flag service is guaranteed to send out in-order updates via websockets? That is to say, in-order with respects to updates of individual feature-flags.
The simple solution for me would be to replace EBP with a Lambda function so that you can push the changes to SNS FIFO and SQS FIFO. The change will result in you having to write a small amount of code for your Lambda compared to your current architecture.