grafanapromqlprometheus-pushgateway

PromQL, Grafana - Count how many times metric's labels values has changed

I'm trying to count how many times Metric_x changed it's labels.

In Prometheus I have 4 metrics: Metric_1, Metric_2, Metric_3, Metric_4.

If I use the query {__name__=~"Metric_\d"}, I get the following output:

In table:

Metric_1{buildURL="job/Onboarding_Test/2/", job="test_test", line="22"} 1
Metric_2{buildURL="job/Onboarding_Test/2/", job="test_test", line="33"} 1
Metric_3{buildURL="job/Onboarding_Test/7/", job="test_test", line="33"} 1 
Metric_4{buildURL="job/Onboarding_Test/2/", job="test_test", line="11"} 1

In Graph's legend:

Metric_1{buildURL="job/Onboarding_Test/1/", job="test_test", line="33"}
Metric_1{buildURL="job/Onboarding_Test/2/", job="test_test", line="22"}
Metric_2{buildURL="job/Onboarding_Test/1/", job="test_test", line="11"}
Metric_2{buildURL="job/Onboarding_Test/2/", job="test_test", line="33"}
Metric_2{buildURL="job/Onboarding_Test/3/", job="test_test", line="22"}
Metric_3{buildURL="job/Onboarding_Test/2/", job="test_test", line="11"}
Metric_3{buildURL="job/Onboarding_Test/6/", job="test_test", line="11"}
Metric_3{buildURL="job/Onboarding_Test/7/", job="test_test", line="33"}
Metric_4{buildURL="job/Onboarding_Test/2/", job="test_test", line="11"}

I want to visualise in Grafana (table view) how many times Metrics changes their labels values. (Changes I can see in "Graph tab" in Prometheus)

Expected output:

name val
Metric_1 2
Metric_2 3
Metric_3 3
Metric_4 1
Solution

  • You can use combination of count and last_over_time:

    count(last_over_time({__name__=~"Metric_\d"}[1d])) by (__name__)

    Here: