promqlbenthos

Defaulting a prometheus counter metric to 0


I'm facing a nightmarishly hard problem to solve for something that seems so trivial. Ultimately, I'm trying to detect a single increment of a counter metric in PromQL when there was no data previously in order to trigger an alert.

I've hit a dead-end in PromQL and the only other solution seems to be to initialise a counter to 0 using Benthos (i.e serve the counter as a value as 0, even though it's not been incremented in the processor pipeline). Is this possible? If it is, it would allow me to detect a change from 0 -> 1 instead of [no-data] -> 1.

Any guidance would be much appreciated.


Solution

  • One way around it is to use a mix of sequence and generate inputs:

    input:
      sequence:
        inputs:
          - generate:
              count: 1
              mapping: root = ""
            processors:
              - metric:
                 type: counter
                 name: test
                 labels:
                   foo: bar
              - mapping: root = deleted()
          - your_actual_input:
              # ...
    

    The message created by the generate input is used to set that test metric and then it's discarded.