amazon-web-servicesaws-lambdaamazon-vpcamazon-kinesisaws-networking

How can my AWS Lambda function inside a private subnet in a VPC consume Kinesis streams without VPC endpoint or NAT Gateway?


Strangely, this is a question why something is working rather than the contrary.

I have a Lambda function triggered by Kinesis events consuming batches of data records. It is used to process and insert data into a database residing within a VPC. After moving the Lambda function into a private subnet of a VPC, I was expecting the connection to Kinesis to be blocked. I have not enabled any VPC endpoint or NAT gateway.

The IAM role is set up to allow connections to this specific Kinesis stream as well as the permissions needed to set up a Lambda in a VPC (AWSLambdaVPCAccessExecutionRole).

The security group the Lambda function blocks all inbound traffic.

I did indeed need to set up a VPC endpoint and allow outbound HTTPS traffic in the Security Group to access the Secrets manager, but for Kinesis it seems to work without this.

How come the Kinesis stream consumption is still working? I was expecting to need a VPC endpoint for this. Is the routing/firewall rules set up under the hood leveraging the IAM settings somehow?


Solution

  • Lambda functions don't have inbound network traffic so inbound security groups are irrelevant.

    Your Lambda function is not polling the Kinesis service or interacting with Kinesis directly in any way so it doesn't need an outbound network route.

    Instead, what is happening is that the AWS Lambda service itself is polling Kinesis on your behalf and the AWS Lambda service is invoking your function, when appropriate, and passing it the Kinesis events. This is a push model from the perspective of your Lambda function, not a pull model.