prometheusprometheus-alertmanager

Prometheus alerts first counter value


I'm trying to create an alert for errors. There is a metric which counts errors occurred in an application. But when I try to catch its increase it always returns 0.

increase(app_error[1h])

Even if I do it with an offset offset 5h, to the point when it was encountered first, it just disappears. Is there a way to find that first error encounter and its first value increasing(from 0 to 1)? I need to fire the alert only on the first count of that metric.


Solution

  • Got it resolved by

    app_error unless app_error offset 1h
    

    return left side of the expression unless it doesn't have matches in the right side of the expression.