I am thinking of using amazon Simpledb as a queue to take some load off of my webservices. The requests are larger than 64k and i need fifo so i cant use SQS. I am wondering what the concurrent limits are on simpledb. can I have 10,000 clients all inserting a new row at roughly the same time?
any help would be great
You'll likely have issues with SimpleDb because of its 'eventual consistency' on writes. This means that when you write data, its not guaranteed to be returned in a query made immediately after. I've heard that you're safe after a couple of seconds, but you have to code the system to take this into account.
This has been mitigated slightly with the introduction of conditional puts, which ensures that there are no lost updates if there are concurrent writes. The eventual consistency model can make it quite complicated to code simple things like counters and queues.
Here is an example of a incrementing counter implimented in Java using the AWS SDK. Notice in the nextValue method:
while (!done) {
Clients essentially retry until their put is cleanly processed. This is not scalable!