I want to create a Kubernetes CronJob that deletes resources (Namespace, ClusterRole, ClusterRoleBinding) that may be left over (initially, the criteria will be "has label=Something" and "is older than 30 minutes". (Each namespace contains resources for a test run).
I created the CronJob, a ServiceAccount, a ClusterRole, a ClusterRoleBinding, and assigned the service account to the pod of the cronjob.
The cronjob uses an image that contains kubectl, and some script to select the correct resources.
My first draft looks like this:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-app
namespace: default
labels:
app: my-app
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: my-app
namespace: default
labels:
app: my-app
spec:
concurrencyPolicy: Forbid
schedule: "*/1 * * * *"
jobTemplate:
# job spec
spec:
template:
# pod spec
spec:
serviceAccountName: my-app
restartPolicy: Never
containers:
- name: my-app
image: image-with-kubectl
env:
- name: MINIMUM_AGE_MINUTES
value: '2'
command: [sh, -c]
args:
# final script is more complex than this
- |
kubectl get namespaces
kubectl get clusterroles
kubectl get clusterrolebindings
kubectl delete Namespace,ClusterRole,ClusterRoleBinding --all-namespaces --selector=bla=true
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: my-app
labels:
app: my-app
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: my-app
subjects:
- kind: ServiceAccount
name: my-app
namespace: default
apiGroup: ""
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: my-app
labels:
app: my-app
rules:
- apiGroups: [""]
resources:
- namespaces
- clusterroles
- clusterrolebindings
verbs: [list, delete]
The cronjob is able to list and delete namespaces, but not cluster roles or cluster role bindings. What am I missing?
(Actually, I'm testing this with a Job first, before moving to a CronJob):
NAME STATUS AGE
cattle-system Active 16d
default Active 16d
fleet-system Active 16d
gitlab-runner Active 7d6h
ingress-nginx Active 16d
kube-node-lease Active 16d
kube-public Active 16d
kube-system Active 16d
security-scan Active 16d
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:default:my-app" cannot list resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): clusterrolebindings.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:default:my-app" cannot list resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:default:my-app" cannot list resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): clusterrolebindings.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:default:my-app" cannot list resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope`
You need to change your ClusterRole like this :
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: my-app
labels:
app: my-app
rules:
- apiGroups: [""]
resources:
- namespaces
verbs: [list, delete]
- apiGroups: ["rbac.authorization.k8s.io"]
resources:
- clusterroles
- clusterrolebindings
verbs: [list, delete]
The ressources are now in the right apiGroup