[SOLVED] Integrate Azure Key Vault With Azure Kubernetes Service using Managed Identity

Integrate Azure Key Vault With Azure Kubernetes Service using Managed Identity

I'm setting up Keyvault integration with k8s in Azure. I can mount a volume with secrets using the csi driver in Azure using Managed identities. I can verify the secret is mounted by exec-ing into the pod and cat-ing out the secrets. However, now I want to expose the secrets as environment variables, but I'm unclear how to do that. Below is the following SecretProviderClass and Pod I have deployed.

spc-keyvault.yaml:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: spc-keyvault
spec:
  provider: azure
  secretObjects:
  - secretName: foobar-secret
    data:
    - key: foobar
      objectName: foobar
    type: Opaque
  parameters:
    keyvaultName: "$keyvault"
    usePodIdentity: "true"
    useVMManagedIdentity: "false"
    userAssignedIdentityID: ""
    cloudName: ""
    objects: |
      array:
        - |
          objectName: foobar
          objectType: secret
          objectVersion: ""
    resourceGroup: "$resourceGroup"
    subscriptionId: "$subId"
    tenantId: "$tenantId"

pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: inject-secrets-from-akv
  labels:
    aadpodidbinding: azure-pod-identity-binding-selector
spec:
  containers:
    - name: nginx
      image: nginx
      env:
      - name: SECRET
        valueFrom:
          secretKeyRef:
            name: foobar-secret
            key: foobar
      volumeMounts:
      - name: secrets-store-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
  volumes:
    - name: secrets-store-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: spc-keyvault

Here's the error when I deploy my Pod manifest:

Events:
  Type     Reason   Age                     From     Message
  ----     ------   ----                    ----     -------
  Warning  Failed   58m (x227 over 108m)    kubelet  Error: secret "foobar-secret" not found
  Normal   Pulling  3m51s (x470 over 108m)  kubelet  Pulling image "nginx"

The error is pretty obvious, there is no secret named: foobar-secret. I thought the csi driver automatically created the required secrets. So after some research and digging throught documentation and source code, I found a suggestion to add nodePublishSecretRef in the pod yaml. Then you need to set nodePublishSecretRef with the aad client id and client secret.[1, 2] However since I'm using managed identity I'm unsure how to do this.

Has anyone got this working using Managed Identity and can provide any insight? Or do i need to create a startup script (via configmap?) to populate the volume mount as env. variables.

Solution

After more digging around, I found this bit to tickle in the helm chart to get the csi driver to create k8s secrets:

secrets-store-csi-driver.syncSecret.enabled = true

Now I have k8s secrets. Figured I would share for anyone else wanting this functionality.