In my current project, I am using AWS AppRunner to expose an HTTP RESTful API (dockerized Python with FastAPI). The AppRunner uses AWS RDS PostgreSQL database, AWS S3, and AWS SES, to send some emails.
In the project's staging environment everything works perfectly, with a fully public AppRunner, and RDS instances. The AppRunner uses a custom IAM role using existent full access policies to access both RDS, S3, and SES, (AmazonRDSDataFullAccess, AmazonS3FullAccess, AmazonSESFullAccess, AWSAppRunnerServicePolicyForECRAccess), in addition to service access to AWS ECR to load the API image from (i.e., latest dockerized Python with FastAPI).
The problem arises when creating the production environment and moving production's database (i.e., AWS RDS PostgreSQL) to a private subnet. As a consequence, the AWS AppRunner's outgoing traffic must be connected to the VPC, and a security group added to allow PostgreSQL traffic connecting to private database's port 5432. An S3 VPC endpoint, with the corresponding security group, is also added to allow connecting AppRunner with AWS S3. The AppRunner VPC Connector, used to connect the AppRunner with the VPC (the only one in the project), is connected to all subnets (both private and public) and uses all the specified security groups. That way, the AppRunner is able to connect and normally use both AWS S3 and RDS database. However, anytime the AppRunner tries to connect with AWS SES it get hungs until a Gateway Timeout is triggered and an HTTP 502 Bad Gateway is obtained.
In order to configure the access from AppRunner, with outgoing traffic to VPC, and the AWS SES, I created first a security group and then a VPC endpoint for SES, as explained [here] (https://docs.aws.amazon.com/ses/latest/dg/send-email-set-up-vpc-endpoints.html), but with no luck. Unfortunatelly, no logs are dumped in the AppRunner once it starts trying to connect with SES, although both my application and boto3 library do. I have tried to, at least test the connectivity from the private subnet to the AWS SES VPC endpoint, using telnet (telnet email-smtp.us-east-1.amazonaws.com 465), confirming that there is connectivity.
Do I have to use an IAM role with another policy, apart from the above specified (including AmazonSESFullAccess) ? Do I have to add any rule to the route table of either my public or private subnets?
I would really appreciate any insight regarding why the AppRunner cannot connect to the AWS SES. Thanks in advance.
P.S: The AWS SES is still in the sandbox.
P.S1: All configuration has been manually done using AWS Console.
P.S2: I had to add a rule to my private subnet's route table to reach AWS S3, using AWS S3 managed prefixed list to route traffic to the AWS S3 VPC endpoint. I tried to do something similar with AWS SES but it seems there is no managed prefixed list (pl-) for AWS SES and CIDR are not allowed to be directly used in route tables with VPC endpoints.
These are my two route tables :
Private subnets Route table
pl-6<HASH1>a vpce-091<HASH2>a
0.0.0.0/0 nat-067<HASH3>3
172.31.0.0/16 local
Public subnets Route table (principal)
0.0.0.0/0 igw-072<HASH4>4
172.31.0.0/16 local
Finally, my problem was related with the NAT Gateway. It seems that I had deleted the original NAT Gateway I had, named nat-067<HASH3>3, and replaced it with a new NAT Gateway, named nat-067<HASH5>5. However, in my private subnet's route table the old the 0.0.0.0/0 entry still referred to the old non-existing NAT Gateway, (i.e., nat-067<HASH3>3). Notice that subnets route table in AWS does check if a referred element (e.g., a NAT Gateway) is still valid.
After noticing the error, and replacing the old 0.0.0.0/0 nat-067<HASH3>3 private subnet0s route table entry with 0.0.0.0/0 nat-067<HASH5>5 everything worked, and my AWS AppRunner Service could properly communicate with AWS SES, sending automated emails as expected.
Detailed explanations about why and how configuring a NAT Gateway in AWS private subnets can be found here.