kubernetesamazon-ekshpahorizontal-pod-autoscaling

Kubernetes Horizontal Pod Autoscaling (HPA) for specific container in the pod


In our deployment, there are 4 containers in the pod. We have defined the resource limits (CPU and memory) only for container-1. The metrics server is installed in the cluster. Our requirement is to apply Horizontal Pod Autoscaling (HPA) with the following requirements. :

When we used following HPA yaml, it checked both cpu and memory. But it consider all containers in the pod. Because when applying, it gave us an error. Therefore, In this method we had to provide resource (cpu, memory) limits for other 3 containers as well. Then it properly showed the percentage of the current memory and cpu usage.

Error:

The HPA was unable to compute the replica count: failed to get cpu utilisation: missing request for cpu in container container-2 of Pod test-pod

apiVersion: autoscaling/v1
  kind: HorizontalPodAutoscaler
  metadata:
    annotations:
      autoscaling.alpha.kubernetes.io/metrics: '[{"type":"Resource","resource":{"name":"memory","targetAverageUtilization":75}}]'
    name: test-hpa
    namespace: ns
  spec:
    maxReplicas: 5
    minReplicas: 2
    scaleTargetRef:
      apiVersion: apps/v1
      kind: Deployment
      name: test-pod
    targetCPUUtilizationPercentage: 65

The issue here is it doesn't only consider container-1 in the pod.

  1. Is there a way that we can give a specific container in the hpa yaml fulfilling the mentioned requirements?
  2. Also, can we apply HPA without providing resource limits (CPU and memory) to the other 3 containers in the pod?

I've tried several HPA yamls but it didn't fulfil the both requirements. Also note that "apiVersion: autoscaling/v2beta2" is not working in our cluster because of the k8 version.


Solution

  • After trying several methods the following code worked, and was able to get both memory and cpu in the pod and the container.

    apiVersion: autoscaling/v1
    kind: HorizontalPodAutoscaler
    metadata:
      annotations:
        autoscaling.alpha.kubernetes.io/metrics: |
          [
            {
              "type": "Resource",
              "resource": {
                "name": "memory",
                "targetAverageUtilization": 75
              }
            },
            {
              "type": "ContainerResource",
              "ContainerResource": {
                "name": "cpu",
                "container": "container-1",
                "targetAverageUtilization": 65
              }
            },
            {
              "type": "ContainerResource",
              "ContainerResource": {
                "name": "memory",
                "container": "container-1",
                "targetAverageUtilization": 75
              }
            }
          ]
      name: test-hpa
      namespace: ns
    spec:
      maxReplicas: 5
      minReplicas: 2
      scaleTargetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: test-pod
      targetCPUUtilizationPercentage: 65
    

    In here the following block in the annotation provide the memory metrics for the pod:

    {
      "type": "Resource",
      "resource": {
        "name": "memory",
        "targetAverageUtilization": 75
      }
    }
    

    CPU metrics for the pod provided by: targetCPUUtilizationPercentage: 65

    And the cpu and memory will be provided by following block in the annotation.

    {
          "type": "ContainerResource",
          "ContainerResource": {
            "name": "cpu",
            "container": "container-1",
            "targetAverageUtilization": 65
          }
        },
        {
          "type": "ContainerResource",
          "ContainerResource": {
            "name": "memory",
            "container": "container-1",
            "targetAverageUtilization": 75
          }
        }
    

    This provides the usage when I describe the hpa.

    kubectl describe hpa -n ns
    

    Metrics:
    ( current / target ) resource memory on pods (as a percentage of request): 63% (5497292117333m) / 75%
    resource cpu of container "container-1" on pods (as a percentage of request): 0% (17m) / 65% resource memory of container "container-1" on pods (as a percentage of request): 124% (2673916586666m) / 75% resource cpu on pods (as a percentage of request): 1% (47m) / 65%

    Thanks for your input @Zhakyp! It helped me resolve some issues. Hope this helps someone else too. This may not be the best way to do this. But only this worked for our k8 version (1.29).