I am trying to make a histogram in Grafana of how much CPU a bunch of pods are using. I don't want an instant snapshot of the current usage, I want it to be an average of how much each pod has used over the last 1 hour.
The problem I'm having is that every query I try seems to return a range vector. For example this query:
sum(sum_over_time(system_cpu_usage{pod=~"$pod"}[1h])) by (pod)
/
sum(count_over_time(system_cpu_usage{pod=~"$pod"}[1h])) by (pod)
returns a range vector like this for each pod
"data": {
"values": [
[
1729874620000,
1729874640000,
1729874660000,
1729874680000,
],
[
0.48956301869585045,
0.6437413841918773,
0.6332958523730499,
0.5662626892438256,
]
]
}
But I just want an instant vector or a scalar value for each pod.
The closest I've been able to get is
last_over_time(avg by (pod) (system_cpu_usage{pod=~"$pod"})[1h:5m])
which returns this for each pod
"data": {
"values": [
[
1729874680000,
],
[
0.5662626892438256,
]
]
}
It's almost exactly what I want. I'd prefer to use avg_over_time but once again, that returns multiple time values.
For what you’re trying to achieve, I think you can use the avg_over_time function but apply it in a way that Grafana interprets as a single, averaged value over the period. More like this:
avg(avg_over_time(system_cpu_usage{pod=~"$pod"}[1h])) by (pod)
This will calculate the average CPU usage over the last hour for each pod and return an instant vector, which should provide one scalar value per pod.
Not sure if this is what you are trying to do but hopefully it helps.