regexprometheuspromql

Reusing a regexp match across 2 metrics/labels in prometheus alerts


For a Prometheus alert I have the following expression:

node_hwmon_tempcelsius{sensor="nvme_nvme0"} >= node_hwmon_temp_crit_celsius{sensor="nvme_nvme0"}

repeated for nvme 0-5. I don't want to enumerate all the nvmes by hand and instead use a regexp sort of like:

node_hwmon_tempcelsius{sensor=~"^nvme_nvme.*$"} >= node_hwmon_temp_crit_celsius{sensor=~"nvme_nvme.*$"}

But whatever matches the first regular expression has to be reused for the second regular expression.

Anyone know the syntax for that?

Full alert definition:

groups:
  - name: 'alert'
    rules:
    - alert: 'NVME Temperature Critical'
      expr: node_hwmon_tempcelsius{sensor=~"^nvme_nvme(.*)$"} >= node_hwmon_temp_crit_celsius{sensor=~"nvme_nvme(.*)$"}
      for: '1m'
      labels:
        severity: 'critical'
      annotations:
        summary: 'Instance {{ $labels.instance }}'
        description: '{{ $labels.instance }} of group {{ $labels.group }} has value {{ $value }}.'

Solution

  • You don't need a second regex at all.

    Prometheus does vector matching, where it applies operations (like >=) only to vectors with same exact label sets (unless instructed otherwise through on or ignoring).

    So in your case, selector sensor=~"nvme_nvme.* applied to a left hand side, will result in right hand side being evaluated only with the same value of label sensor.
    As a result, you can just use query

    node_hwmon_tempcelsius{sensor=~"nvme_nvme.*"} >= node_hwmon_temp_crit_celsius
    

    in your alert, and promQL will handle what is needed for you.


    In some cases regex in the RHS can makes some sense - for example, to limit number of considered time series. But this is not the case for such a simple query as you've shown. And even then no additional regex logic would be required, as vector matching still would be in place.