We use kafka chart (version 17) as a dependency of our chart. And for I try to configure monitoring of that kafka.
values.yaml:
kafka:
metrics:
kafka:
enabled: true
jmx:
enabled: true
serviceMonitor:
enabled: true
labels:
my.custom.label/service-monitor: "1.0"
...
Then I deploy my application and go to prometheus targets and I see following:
I am concerned about the failed request.
What I see ?
There are 2 services to monitor:
and both of those services are requested by 2 paths:
But kafka-exporter
doesn't respond on /
. Is there way to configure kafka chart to work without failures if there are 2 exporters are enabled ?
Services definition:
spec:
endpoints:
- path: /
port: http-metrics
namespaceSelector:
matchNames:
- my-ns
selector:
matchLabels:
app.kubernetes.io/component: metrics
app.kubernetes.io/instance: dev-my-app
app.kubernetes.io/name: my-app-kafka
spec:
endpoints:
- path: /metrics
port: http-metrics
namespaceSelector:
matchNames:
- my-ns
selector:
matchLabels:
app.kubernetes.io/component: metrics
app.kubernetes.io/instance: dev-my-app
app.kubernetes.io/name: my-app-kafka
Eventually I finished with following solution:
I use jmx exporter from kafka chart:
values.yaml
kafka:
metrics:
kafka:
enabled: false
jmx:
enabled: true
serviceMonitor:
enabled:true
labels:
my.super.company/service-monitor: "1.0"
enabled: true
and registered kafka exporter as separated dependency:
chart.yaml:
- name: prometheus-kafka-exporter
version: 1.6.0
repository: https://prometheus-community.github.io/helm-charts
and configured it manually:
values.yaml
prometheus-kafka-exporter:
kafkaServer:
- my-kafka-server:9092
prometheus:
serviceMonitor:
enabled: true
namespace: ""
additionalLabels:
my.super.company/service-monitor: "1.0"