amazon-web-servicesamazon-ec2aws-cdkamazon-ebs

EC2 AutoScalingGroup leaving EBS volumes around after instance termination


I have an auto scaling group that uses an AMI that has EBS volumes associated with it. Every time that the group scales those EBS volumes get recreated.

Also due to reasons, i have a lifetime limit on the scaling group servers of 24 hours.

I've found that while the EC2 instance cleans itself up well, it leaves the EBS volumes in a detached state. Over the course of the month this has caused my EBS storage costs to explode because each volume created is 1tb in size.

I'm using CDK to create my scaling group and i can't seem to find any way to set the retention policy on the resources created.

Anyone have a way to have the EBS volumes destroyed upon instance termination?

        const scalingGroup = new AutoScalingGroup(this, 'ServerScalingGroup', {
            vpc,
            role,
            securityGroup,
            minCapacity: 2,
            maxCapacity: 20,
            requireImdsv2: true,
            instanceType: InstanceType.of(InstanceClass.C7I, InstanceSize.XLARGE),
            machineImage: MachineImage.lookup({
                windows: true,
                name: options.ami
            }),
            healthCheck: HealthCheck.elb({
                grace: cdk.Duration.minutes(5)
            }),
            maxInstanceLifetime: cdk.Duration.days(1),
            cooldown: cdk.Duration.minutes(2)
        });

I see two potential solutions here using notifications or using addLifecycleHook() but both of those seem like hacks.


Solution

  • If the block devices setting is explicitly set in the CDK (or CF), the setting from the AMI is ignored. Here's the relevant yaml from a a CF stack for a vendor-provided AMI:

      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            DeleteOnTermination: true
            VolumeSize: 2000
    

    Those settings can be assigned in CDK via the ASG, Instance, or LaunchTemplate resources.