I ran into this problem and solved it, so this Q&A is here just in case somebody else spent time on sifting through AWS, Keda, and/or Kubernetes' docs trying to deduce this answer
My team deployed Keda with the goal of horizontally autoscaling pods in Kubernetes based on Redis queue length, CPU utilization, and memory utilization. Post-deployment, we noticed the horizontal pod autoscaler as viewed in ArgoCD was throwing the error:
unable to get metrics for resource memory: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
Some other errors we saw:
$ kubectl get --raw "/apis/metrics.k8s.io/v1beta1"
Error from server (NotFound): the server could not find the requested resource
$ kubectl top nodes
error: Metrics API not available
Of course, these errors are in addition to the human-visible issue of pods not autoscaling when CPU/Memory utilization thresholds are reached.
This occurs despite the redis queue pod scaling as expected via Keda/HPA.
What can we do to make sure the CPU and Memory utilization causes scaling as expected?
As it turns out, this is caused by:
metrics-server under the hood for CPU/Memory utilization metrics; this is not noted on the CPU and Memory utilization pages in their documentation, but rather is noted in a sidebar on their Keda metrics querying page (as of this writing, https://keda.sh/docs/2.8/operate/metrics-server/) that reads: Note: There are 2 exceptions in querying metrics and those are cpu and memory scalers. When KEDA creates the HPA object, it uses standard cpu and memory metrics from the Kubernetes Metrics Server. If you want to query these 2 specific values, you should do it using /apis/metrics.k8s.io/v1beta1 instead of /apis/external.metrics.k8s.io/v1beta1metrics-server by default. For reference, as of this writing AWS docs read "In Amazon EKS, Metrics Server isn't installed by default." (https://aws.amazon.com/premiumsupport/knowledge-center/eks-metrics-server/)To resolve this, we installed metrics-server via its Helm chart to our cluster.