How to keep the same public IP addresses for a set of AWS EC2 instances with Terraform

I have a terraform config that creates 4 EC2 instances and gives them static public IP addresses:

resource "aws_instance" "kit-prod-api" {
  subnet_id                   = element(module.vpc.public_subnets, count.index)
  ami                         = data.aws_ami.kit_prod_api_ami.id
  instance_type               = "c5.large"
  associate_public_ip_address = true
  vpc_security_group_ids      = [module.vpc.default_security_group_id, aws_security_group.prod_ssh_security_group.id, aws_security_group.prod_lb_instances_security_group.id]
  count                       = 4
  lifecycle {
    create_before_destroy = "true"
    ignore_changes = [
      ami,
      instance_type,
    ]
  }
  # there are tags and provisioner blocks here too
}

When I want to update the instances to use a new AMI, I terminate one at a time and run terraform apply to replace them. This works, but they get new public IP addresses. I can see that their old IP addresses show up in the EIP section of the AWS console, but they weren't there beforehand. Is there a way to modify this script/workflow so that the same IP addresses get associated with the new instances?

Solution

IP addresses in EC2 belong to network interfaces rather than to EC2 instances, but the EC2 API allows implicitly creating a network interface as a side-effect of creating an EC2 instance and so it often appears as though network interface attributes like IP addresses belong directly to EC2 instances, even though that isn't strictly true.

You can make a network interface outlive the EC2 instance it's attached to by declaring the network interface as a separate resource:

resource "aws_network_interface" "example" {
  count = 4

  subnet_id = element(module.vpc.public_subnets, count.index)
}

resource "aws_instance" "example" {
  count = length(aws_network_interface.example)

  # ...

  network_interface {
    network_interface_id = aws_network_interface.example[count.index].id
  }
}

The separation above allows the private IPv4 address associated with the network interface to survive even if the associated EC2 instances are replaced. However, note that only one EC2 instance can be attached to each network interface at a particular time, so you won't be able to use create_before_destroy in this case -- that would cause two instances to try to attach to the same network interface at the same time -- and so you'll need to find a different strategy to ensure continuity in situations where you're replacing all of the instances.

The above only causes the private IP address to be preserved. To associate a consistent public IP address with each network interface you'll need to additionally declare an Elastic IP address, which allows you to allocate a public IP address whose lifetime is independent of any particular network interface or EC2 instance. The Elastic IP address will be associated with the network interface so it can also survive any particular instance being replaced:

# (this is in addition to the example above)

resource "aws_eip" "example" {
  count = length(aws_network_interface.example)

  network_interface = aws_network_interface.example[count.index].id
}

To avoid the public IP address of the EC2 instance changing after it's already started booting, you should also tell Terraform about the hidden dependency between the EC2 instance resource and the elastic IP address resource, since Terraform cannot infer it automatically from references:

resource "aws_instance" "example" {
  # everything as before, plus:
  depends_on = [aws_eip.example]
}

More information on these resource types: