I'm doing some POC for security research, trying to access namespace secrets directly from a worker node. I have a cluster on GKE running Kubernetes 1.20
I'm running the following command from a worker (none-master) node:
curl -v $APISERVER/api/v1/namespaces/default/pods/ \
--cacert /etc/srv/kubernetes/pki/ca-certificates.crt \
--cert /var/lib/kubelet/pki/kubelet-client.crt \
--key /var/lib/kubelet/pki/kubelet-client.key
And it works fine.
However, trying to get secrets fails:
curl -v $APISERVER/api/v1/namespaces/default/secrets/ \
--cacert /etc/srv/kubernetes/pki/ca-certificates.crt \
--cert /var/lib/kubelet/pki/kubelet-client.crt \
--key /var/lib/kubelet/pki/kubelet-client.key
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "secrets is forbidden: User \"system:node:gke-XXX--YYY\" cannot list resource \"secrets\" in API group \"\" in the namespace \"pencil\": No Object name found",
"reason": "Forbidden",
"details": {
"kind": "secrets"
},
"code": 403
Looking at the documentation, I see that kubelet running on node should be able to access secrets: https://kubernetes.io/docs/reference/access-authn-authz/node/
And from my understanding, the authorization is backed by the ClusterRole
system:node
. Looking at it I do see it has the role to get
secrets:
% kubectl get clusterrole system:node -o json
{
"apiVersion": "rbac.authorization.k8s.io/v1",
"kind": "ClusterRole",
...
{
"apiGroups": [
""
],
"resources": [
"configmaps",
"secrets"
],
"verbs": [
"get",
"list",
"watch"
]
},
...
]
}
And some more relevant documentation for communication between kubelet and kube-apiserver: https://kubernetes.io/docs/concepts/architecture/control-plane-node-communication/#node-to-control-plane
After digging in the source code, I found the meaning of No Object name found
error - secrets or configmaps must be named in order to be retrieved. As suggested by the documentation, they can be retrieved if mapped to the node in question by some pod.
Therefore assuming some secret server-password
is used by some pod in my node, following command worked as expected:
curl -v $APISERVER/api/v1/namespaces/default/secrets/server-password \
--cacert /etc/srv/kubernetes/pki/ca-certificates.crt \
--cert /var/lib/kubelet/pki/kubelet-client.crt \
--key /var/lib/kubelet/pki/kubelet-client.key
Also apparently kubectl
can be used just as simply, its the one creating the certificates at /var/lib/kubelet/pki
in the first place.
kubectl --kubeconfig /var/lib/kubelet/kubeconfig -n default get secret server-password -o json