aws-lambdaamazon-sqsamazon-snsamazon-sesamazon-cloudwatchlogs

How to send large number of mails (more than sending rate limit) at a time in AWS SES

I have an AWS SES service with a sending rate of 100 mails per second. Our application sends emails to all students within educational institutes. There's a need to distribute a substantial volume of emails, such as 1000 at a time, to inform all the students. However, we've encountered issues when attempting to send all the emails at a time, resulting in non-delivery. The number of mails to be sent may vary.

  1. I'm seeking guidance on how to efficiently send a large number of emails within a limited timeframe without experiencing rejections or bounces, utilizing other AWS services.

  2. I would like to get a feedback in our application listing rejected mails, so how to find the bounced or rejected mails

  3. is there any other way to achieve this using lambda/sqs

Solution

  • Assuming the total throughput quota is adequate to eventually send all your emails, and you just need a way to smooth the load curve, you should consider a system that queues emails and sends them at a fixed rate.

    Here's an example of how you could implement a rate-limited email queue:

    1. Add your emails to an SQS queue
    2. Poll the queue at a fixed interval, pulling a batch of messages sized such that it will not exceed the service limits
    3. Send the emails immediately using SES
    4. Configure SES to publish email delivery status to a Kinesis stream
    5. Consume the Kinesis stream from a Lambda trigger as messages arrive, and store bounces/rejections in your system

    Polling the queue at a fixed interval can be tricky. The simplest solution may be an EventBridge schedule which triggers a Lambda every minute; that Lambda could (2) pull emails from the queue in compliance with your SES quota. However EventBridge schedules don't support an interval shorter than 1 minute.

    If you need a shorter interval (eg for login codes that are expected to be delivered quickly), you could add an intermediate such as a step function which would trigger the Lambda several times. In this case your EventBridge schedule could trigger the step function every minute, which would result in multiple Lambda triggers each minute.

    If you calculate your timing and batch sizes correctly, this should result in a system that cannot exceed the SES rate limit.