I have installed Prometheus-adapter along with the default metrics-server that comes with k3s securely on port 443.
Unfortunately, I get no resources when I query custom.metrics.k8s.io
$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq .
{
"kind": "APIResourceList",
"apiVersion": "v1",
"groupVersion": "custom.metrics.k8s.io/v1beta1",
"resources": []
}
When I look at the logs of Prometheus-adapter I get unable to update list of all metrics: unable to fetch metrics for query ...: x509: certificate is valid for localhost, localhost, not metrics-server.kube-system
How can I resolve this issue?
To solve this issue, I had to create separate certificate for both metrics-server and adapter. Adapter also has an issue about adding capability to ignore cert validation which wasn't merged.
For metrics-server and cert request I used the following:
{
"hosts": [
"prometheus-adapter",
"prometheus-adapter.monitoring",
"prometheus-adapter.monitoring.svc",
"prometheus-adapter.monitoring.pod",
"prometheus-adapter.monitoring.svc.cluster.local",
"prometheus-adapter.monitoring.pod.cluster.local",
"<pod ip>",
"<service ip>"
],
"CN": "prometheus-adapter.monitoring.pod.cluster.local",
"key": {
"algo": "ecdsa",
"size": 256
},
}
{
"hosts": [
"metrics-server",
"metrics-server.kube-system",
"metrics-server.kube-system.svc",
"metrics-server.kube-system.pod",
"metrics-server.kube-system.svc.cluster.local",
"metrics-server.kube-system.pod.cluster.local",
"<service ip>",
"<pod ip>"
],
"CN": "metrics-server.kube-system",
"key": {
"algo": "ecdsa",
"size": 256
},
}
For ca, you can create your certificate authority or use Kubernetes signers as indicated here The only point worth noting here is that if you use either of signers, you should mount the ca bundle yourself to your deployments.
Finally, mount tls keys and ca bundle to your deployment.
extraArguments:
- --tls-cert-file=/var/run/serving-cert/tls.crt
- --tls-private-key-file=/var/run/serving-cert/tls.key
- --client-ca-file=/etc/ssl/certs/ca.crt