kubernetesopenshiftprometheuspromql

How to count how many pods were started by namespace each day in PromQL?


I'm trying to answer a question of how many pods were started/scheduled/whatever per namespace per day. I have not found any useful counter-type metric that would be counting that, just related gauges like kube_pod_status_scheduled_time.

So far best thing I was able to come up with is this:

topk(3,
  sum_over_time((
    delta((
      count by (namespace) (
        count by (pod, namespace) (kube_pod_status_scheduled_time{namespace=~".*-tenant"})
      )
    )[5m:]) > 0
  )[1d:])
)

But first this seems to be too heavy/inefficient to finish before request timeout and second I'm very skeptical it is actually correct.

Would you have any ideas please?


Solution

  • Try the following query:

    count(
      last_over_time(kube_pod_created[24h]) > time()-24*3600
    ) by (namespace)
    

    It should return the number of per-namespace pods, which were created during the last 24 hours.

    See the docs about kube_pod_created metric here.

    The last_over_time function is needed in order to take into account short-lived pods, which were created and stopped during the last 24 hours.

    The comparison with time() - 24*3600 is needed in order to filter out pods, which were started earlier than 24 hours ago.

    The count function just counts the number of the selected time series.