kubernetespolicies

Kyverno ClusterPolicy exception not working as expected while verifying ocisourcelabels


Created the below cluster policy which blocks all images which does not have com.*****.os-policy.verified in their docker label and this is working as expected.

apiVersion: kyverno.io/v2beta1
kind: ClusterPolicy
metadata:
  name: disallow-host-namespaces
spec:
  validationFailureAction: Enforce
  background: false
  rules:
    - name: verify-oci-annotation
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Images must specify a source/base image from which they are built."
        foreach:
          - list: "request.object.spec.containers"
            context:
              - name: imageData
                imageRegistry:
                  reference: "{{ element.image }}"
              - name: ocisourcelabels
                variable:
                  jmesPath: imageData.configData.config.Labels | keys(@)
                  default: []
            deny:
              conditions:
                all:
                  - key: "com.*****.os-policy.verified"
                    operator: AnyNotIn
                    value: "{{ ocisourcelabels}}"

But when I tried to create a PolicyException to skip the policy for deployments with specific annotations its not working.

apiVersion: kyverno.io/v2alpha1
kind: PolicyException
metadata:
  name: delta-exception
  namespace: test
spec:
  exceptions:
    - policyName: disallow-host-namespaces
      ruleNames:
        - verify-oci-annotation
        - autogen-host-namespaces
  match:
    any:
      - resources:
          kinds:
            - Pod
            - Deployment
          namespaces:
            - test

The below deployment I tried after applied the Policyexcemption but it will fail

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
  annotations:
    *****.com-os-policy-internal-use: "true"
    *****.com-os-policy-break-glass: "true"

spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: junkimagename:latest
          ports:
            - containerPort: 80

But with the same Policyexcemption if I tried to create a pod it will work

apiVersion: v1
kind: Pod
metadata:
  name: badpod
  namespace: default
  annotations:
    *****.com/os-policy.internal-use: "true"
    *****.com/os-policy.break-glass: "true"
spec:
  automountServiceAccountToken: false
  containers:
    - image: junkimagename:latest
      name: node-exporter

How to write a PolicyException here in this scenario which can allow the kind:Deployment to work if specifc annotations are provided


Solution

  • Could fix this with a preconditions. This helped in removing the need to add an excemptionpolicy

    In precondtion I am checking if there is an annotation defined for the image then skip this policy.

    Example:

    apiVersion: kyverno.io/v1
    kind: Policy
    metadata:
      name: kyverno-policy
      namespace: test
    spec:
      validationFailureAction: Enforce
      background: true
      rules:
        - name: verify-oci-annotation
          match:
            any:
              - resources:
                  kinds:
                    - Pod
          preconditions:
            all:
              - key: "{{ request.\"object\".metadata.annotations.\"*****.com/os-policy.internal-use\" || '' }}"
                operator: NotEquals
                value: "true"
          validate:
            message: "Images must specify a source/base image from which they are built."
            foreach:
              - list: "request.object.spec.containers"
                context:
                  - name: imageData
                    imageRegistry:
                      reference: "{{ element.image }}"
                  - name: ocisourcelabels
                    variable:
                      jmesPath: imageData.configData.config.Labels | keys(@)
                      default: []
                deny:
                  conditions:
                    all:
                      - key: "com.******.os-policy.verified"
                        operator: AnyNotIn
                        value: "{{ ocisourcelabels}}"