labelprometheusgrafanametricsgrafana-alerts

Firing many alerts from one rule by metric label

There is a way to fire many alerts from same rule using the metric label?

I have a prometheus counter metric with a label “client”.

How I configure one rule to fire for each client that separately satisfy a fire condition?

My version is 8.4.2

Solution

  • This is exactly the way alerts work in Prometheus. It will generate one alert for each label combination that satisfies a fire condition.

    For example, the following rule:

      - alert: InstanceIsDown
    expr: probe_success{job="blackbox"} == 0
    annotations:
      summary: 'Instance {{ $labels.instance }} is down'

    The Blackbox "probe_success" metric has an "instance" label. If the instances "xxx" and "yyy" are down, the rule will generate two alerts, one for each instance.