I have an ECS stack made with AWS CDK.
import * as cdk from 'aws-cdk-lib'
import { Construct } from 'constructs'
import * as ec2 from 'aws-cdk-lib/aws-ec2'
import * as ecs from 'aws-cdk-lib/aws-ecs'
import path = require('path')
export class CdkPlaygroundEcsAuroraStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props)
// VPC
const vpc = new ec2.Vpc(this, 'ecs-vpc', {
maxAzs: 2,
subnetConfiguration: [
{
name: 'public',
subnetType: cdk.aws_ec2.SubnetType.PUBLIC
},
{
name: 'private',
subnetType: cdk.aws_ec2.SubnetType.PRIVATE_ISOLATED
}
]
})
// VPC Endpoints
const ecrVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'ECRVpcEndpoint', {
vpc,
service: ec2.InterfaceVpcEndpointAwsService.ECR,
privateDnsEnabled: true
})
const cloudWatchVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'cloudWatchVpcEndpoint', {
vpc,
service: ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH,
privateDnsEnabled: true
})
const s3GatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'S3GatewayEndpoint', {
service: ec2.GatewayVpcEndpointAwsService.S3,
vpc,
subnets: [{ subnetType: cdk.aws_ec2.SubnetType.PRIVATE_ISOLATED }]
})
// ECS Cluster
const ecsCluster = new ecs.Cluster(this, 'ecs-cluster', {
vpc,
enableFargateCapacityProviders: true,
clusterName: 'ecs-cluster'
})
const taskDefinition = new ecs.FargateTaskDefinition(this, 'task1', {
cpu: 256,
memoryLimitMiB: 512
})
const container = taskDefinition.addContainer('container', {
image: ecs.ContainerImage.fromAsset(path.resolve(__dirname, '../server')),
memoryLimitMiB: 256
})
const service = new ecs.FargateService(this, 'server', {
cluster: ecsCluster,
taskDefinition,
serviceName: 'server',
vpcSubnets: {
subnetType: cdk.aws_ec2.SubnetType.PRIVATE_ISOLATED
}
})
}
}
It contains an ECS Task running in a private subnet without a NAT Gateway. I want to pull the image via the configured VPC Interface Endpoint. According to the documentation, I have setup VPC Endpoints for ECR, S3 and Cloudwatch.
But the task can not pull the image for the container with the following error:
CannotPullContainerError: pull image manifest has been retried 5 time(s): failed to resolve ref 650289367947.dkr.ecr.eu-central-1.amazonaws.com/cdk-hnb659fds-container-assets-650289367947-eu-central-1:d71b37a3ce0b63a08136ce5b816ea2d5d4b677fc8e3ee8b6eda9738d7c16ebb1: failed to do request: Head "https://650289367947.dkr.ecr.eu-central-1.amazonaws.com/v2/cdk-hnb659fds-container-assets-650289367947-eu-central-1/manifests/d71b37a3ce0b63a08136ce5b816ea2d5d4b677fc8e3ee8b6eda9738d7c16ebb1": dial tcp 3.121.190.14:443: i/o timeout
To me it looks like ECS is trying to access ECR over the internet (dial tcp 3.121.190.14:443). Why doesn't ECS use the VPC Interface Endpoint and how can I change this to fix the error?
To access ECR from ECS via VPC Endpoints, you require 3 endpoints. If you are using Fargate Version > 1.4.0, you will need the following endpoints:
com.amazonaws.region.ecr.dkr
to access the Docker Registry APIscom.amazonaws.region.ecr.api
to access the Amazon ECR APIcom.amazonaws.region.s3
to download the images from S3If you also wish to enable CloudWatch logging, you will additionally need the com.amazonaws.region.logs
endpoint.
The complete cdk definition would look like this:
// access ECR
new ec2.InterfaceVpcEndpoint(this, 'ECRVpcEndpoint', {
vpc,
service: ec2.InterfaceVpcEndpointAwsService.ECR,
privateDnsEnabled: true
})
new ec2.InterfaceVpcEndpoint(this, 'ECRDockerVpcEndpoint', {
vpc,
service: ec2.InterfaceVpcEndpointAwsService.ECR_DOCKER,
privateDnsEnabled: true
})
new ec2.GatewayVpcEndpoint(this, 'S3GatewayEndpoint', {
service: ec2.GatewayVpcEndpointAwsService.S3,
vpc,
subnets: [{ subnetType: cdk.aws_ec2.SubnetType.PRIVATE_ISOLATED }]
})
// access Cloudwatch logging
new ec2.InterfaceVpcEndpoint(this, 'CloudWatchLogsVpcEndpoint', {
vpc,
service: ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
privateDnsEnabled: true
})