I am trying to expose the Prometheus metric that shows LAG value in each Kafka topic. I use kafka_exporter for exporting Kafka metrics.
I added this pattern in prom-jmx-agent.config.yaml
lowercaseOutputName: true
rules:
- pattern: kafka.consumer<type=consumer-fetch-manager-metrics, client-id=(.+), topic=(.+), partition=(.+)><>(records-lag)
name: kafka_consumer_fetch_records_lag
labels:
clientId: "$1"
topic: "$2"
partition: "$3"
help: "Kafka Consumer Fetch Records Lag"
type: GAUGE
I create some lags to be able to see in the Prometheus
I expect to see metric named kafka_consumer_fetch_records_lag
but it does not appear in the metric list.
The above JMX exporter config must be added to JVM-based consumer clients. The brokers don't have kafka_consumer_fetch_records_lag
metric
If you want to find lag of all clients (including non JVM ones),