I can see we can over-provision in EKS for low latecy & I find "ASG Warm pools" similar. My question is, how relevant "ASG Warm Pools" are instead of "Over-provisioning" in AWS EKS?
We use both where I work currently:
ASG Warm Pools allow you to do some sort of initialization, then keep the EC2 Instances in a "Stopped" state. That means that they'll be relatively fast to boot compared with instances launching completely cold.
Overprovisioned instances are "hot" instances, meaning that you spend the money running the compute, but they are ready to receive Pods (they are discoverable by kubectl get nodes
).
The tradeoff is between cost (overprovisioned is more $$$) vs. speed (overprovisioned Nodes are part of the cluster, and will show up as "Ready", compared with Warm Nodes, which will need to be turned on by the Autoscaler signaling to increase the ASG desired count.)
The reason we use both is that we want the benefit of having "hot" instances, with images already pulled on them and ready to go, plus if we need to scale up beyond our overprovisioned counts, we want to be able to quickly add Nodes into the cluster.