Kubernetes Rest API node CPU and RAM usage in percentage

I would like to develop a simple webservice, which returns kubernetes metrics like node / pod CPU and RAM usage in percentage. In CLI there are commands like kubectl top node or kubectl describe node with all the metrics i need. However i am unable find such metrics in kubernetes REST api endpoints. The closest one i got is /apis/metrics.k8s.io/v1beta1/nodes endpoint, which shows CPU and RAM usage but no in percentage.

Is there such an endpoint which show the same information as in top or describe commands or, perhaps, there are better way than metrics.k8s.io addon.

Solution

When you issue kubectl top node, kubectl makes multiple HTTP requests to multiple endpoints. You can see exact endpoints by adding --v=9 flag to kubectl.

For example, in my case

kubectl top node gke-cluster-1-default-pool-99238d56-bv6z --v=9
[...]
I0726 10:41:18.347144    1986 round_trippers.go:435] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.21.0 (linux/amd64) kubernetes/cb303e6" 'https://<ip-address>/apis/metrics.k8s.io/v1beta1/nodes/gke-cluster-1-default-pool-99238d56-bv6z'
I0726 10:41:18.489068    1986 round_trippers.go:435] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.21.0 (linux/amd64) kubernetes/cb303e6" 'https://<ip-address>/api/v1/nodes/gke-cluster-1-default-pool-99238d56-bv6z'
[...]

(there are a lot more, those two are important to answer your question)

First request return

{
    "kind":"NodeMetrics",
    "apiVersion":"metrics.k8s.io/v1beta1",
    "metadata":{
        "name":"gke-cluster-1-default-pool-99238d56-bv6z",
        "selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/gke-cluster-1-default-pool-99238d56-bv6z",
        "creationTimestamp":"2021-07-26T08:41:19Z"
    },
    "timestamp":"2021-07-26T08:41:07Z",
    "window":"30s",
    "usage":{
        "cpu":"86855567n",
        "memory":"950228Ki"
    }
}

.usage.cpu and .usage.memory show used CPU and memory respectively.

Second request return (truncated, the response is huge)

{
    "status":{
        "capacity":{
            "attachable-volumes-gce-pd":"15",
            "cpu":"2",
            "ephemeral-storage":"98868448Ki",
            "hugepages-1Gi":"0",
            "hugepages-2Mi":"0",
            "memory":"4031624Ki",
            "pods":"110"
        },
        "allocatable":{
            "attachable-volumes-gce-pd":"15",
            "cpu":"940m",
            "ephemeral-storage":"47093746742",
            "hugepages-1Gi":"0",
            "hugepages-2Mi":"0",
            "memory":"2885768Ki",
            "pods":"110"
        }
    }
}

.status.allocatable.cpu and .status.allocatable.memory show how much CPU and memory can be allocated to running pods.

As you can see, there is no endpoint that return usage in percentage, kubectl does the calculation on the fly, to output the result in human friendly format

$ kubectl top node gke-cluster-1-default-pool-99238d56-bv6z
NAME                                       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
gke-cluster-1-default-pool-99238d56-bv6z   108m         11%    928Mi           32%