I'm just starting with Kubernetes, and I'm working with HPA (HorizontalPodAutoscaler):
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: find-complementary-account-info-1
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: find-complementary-account-info-1
minReplicas: 2
maxReplicas: 5
metrics:
- type: Resource
resource:
name: memory
target:
type: AverageValue
averageUtilization: 50
But when I try to see the metrics on the console, it doesn't show me:
$ kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
find-complementary-account-info-1 Deployment/find-complementary-account-info-1 <unknown>/50% 2 5 2 143m
My manifest is:
apiVersion: apps/v1
kind: Deployment
metadata:
name: find-complementary-account-info-1
labels:
app: find-complementary-account-info-1
spec:
replicas: 2
selector:
matchLabels:
app: find-complementary-account-info-1
template:
metadata:
labels:
app: find-complementary-account-info-1
spec:
containers:
- name: find-complementary-account-info-1
image: find-complementary-account-info-1:latest
imagePullPolicy: IfNotPresent
resources:
limits:
memory: "350Mi"
requests:
memory: "300Mi"
ports:
- containerPort: 8083
env:
- name: URL_CONNECTION_BD
value: jdbc:oracle:thin:@10.161.6.15:1531/DEFAULTSRV.WORLD
- name: USERNAME_CONNECTION_BD
valueFrom:
secretKeyRef:
name: credentials-bd-pers
key: user_pers
- name: PASSWORD_CONNECTION_BD
valueFrom:
secretKeyRef:
name: credentials-bd-pers
key: password_pers
apiVersion: v1
kind: Service
metadata:
name: svc-find-complementary-account-info-1
labels:
app: find-complementary-account-info-1
namespace: default
spec:
selector:
app: find-complementary-account-info-1
type: LoadBalancer
ports:
-
protocol: TCP
port: 8083
targetPort: 8083
nodePort: 30025
externalIPs:
- 10.161.174.68
Show me the <unknown> tag
Also, if I perform a kubectl describe hpa, it throws me the following:
$ kubectl describe hpa find-complementary-account-info-1
Name: find-complementary-account-info-1
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Thu, 29 Oct 2020 13:57:58 -0400
Reference: Deployment/find-complementary-account-info-1
Metrics: ( current / target )
resource memory on pods (as a percentage of request): <unknown> / 50%
Min replicas: 2
Max replicas: 5
Deployment pods: 2 current / 0 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource memory: no metrics returned from resource metrics API
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 4m49s (x551 over 144m) horizontal-pod-autoscaler unable to get metrics for resource memory: no metrics returned from resource metrics API
I am not working in the cloud, I am working in an on-premise environment configured with Bare-Metal
metrics-server is installed:
$ kubectl get deployment metrics-server -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 1/1 1 1 177m
What's missing? Could someone give me a hand?
$ kubectl describe pod metrics-server -n kube-system
Name: metrics-server-5f4b6b9889-6pbv8
Namespace: kube-system
Priority: 0
Node: tmp224/10.164.21.169
Start Time: Thu, 29 Oct 2020 13:27:19 -0400
Labels: k8s-app=metrics-server
pod-template-hash=5f4b6b9889
Annotations: cni.projectcalico.org/podIP: 10.244.119.140/32
cni.projectcalico.org/podIPs: 10.244.119.140/32
Status: Running
IP: 10.244.119.140
IPs:
IP: 10.244.119.140
Controlled By: ReplicaSet/metrics-server-5f4b6b9889
Containers:
metrics-server:
Container ID: docker://f71d26dc2c8e787ae9551faad66f9588a950bf0a6d0d5cb90ff11ceb219e9b37
Image: k8s.gcr.io/metrics-server-amd64:v0.3.6
Image ID: docker-pullable://k8s.gcr.io/metrics-server-amd64@sha256:c9c4e95068b51d6b33a9dccc61875df07dc650abbf4ac1a19d58b4628f89288b
Port: 4443/TCP
Host Port: 0/TCP
Args:
--cert-dir=/tmp
--secure-port=4443
State: Running
Started: Thu, 29 Oct 2020 13:27:51 -0400
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/tmp from tmp-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from metrics-server-token-4mn92 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
tmp-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
metrics-server-token-4mn92:
Type: Secret (a volume populated by a Secret)
SecretName: metrics-server-token-4mn92
Optional: false
QoS Class: BestEffort
Node-Selectors: kubernetes.io/arch=amd64
kubernetes.io/os=linux
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events: <none>
The error unable to get metrics for resource memory: no metrics returned from resource metrics API is not caused by faulty HPA but by metrics-server that is not able to scrape any metrics.
Most of the time this error is caused by missing commands in metrics-server deployment. It can be done by adding following arguments in the deployment.
--kubelet-insecure-tls
--kubelet-preferred-address-types=InternalIP,ExternalIP