dockerkubernetesenvironment-variableskubernetes-secretscsi

Clarification on the security of using secretKeyRef in Kubernetes manifest


I was looking into an entirely separate issue and then came across this question which raised some concerns:

https://stackoverflow.com/a/50510753/3123109

I'm doing something pretty similar. I'm using the CSI Driver for Azure to integrate Azure Kubernetes Service with Azure Key Vault. My manifests for the integration are something like:

apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentity
metadata:
  name: aks-akv-identity
  namespace: prod
spec:
  type: 0                                 
  resourceID: $identityResourceId
  clientID: $identityClientId
---
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentityBinding
metadata:
  name: aks-akv-identity-binding
  namespace: prod
spec:
  azureIdentity: aks-akv-identity
  selector: aks-akv-identity-binding-selector
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: aks-akv-secret-provider
  namespace: prod
spec:
  provider: azure
  secretObjects:
  - secretName: ${resourcePrefix}-prod-secrets
    type: Opaque
    data:
    - objectName: PROD-PGDATABASE
      key: PGDATABASE
    - objectName: PROD-PGHOST
      key: PGHOST
    - objectName: PROD-PGPORT
      key: PGPORT
    - objectName: PROD-PGUSER
      key: PGUSER
    - objectName: PROD-PGPASSWORD
      key: PGPASSWORD
  parameters:
    usePodIdentity: "true"                                        
    keyvaultName: ${resourceGroupName}akv
    cloudName: ""                               
    objects: |
      array:
            objectName: PROD-PGDATABASE             
            objectType: secret                 
            objectVersion: ""
        - |
            objectName: PROD-PGHOST             
            objectType: secret                 
            objectVersion: ""
        - |
            objectName: PROD-PGPORT             
            objectType: secret                 
            objectVersion: ""
        - |
            objectName: PROD-PGUSER             
            objectType: secret                 
            objectVersion: ""
        - |
            objectName: PROD-PGPASSWORD             
            objectType: secret                 
            objectVersion: ""
    tenantId: $tenantId

Then in the micro service manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment-prod
  namespace: prod
spec:
  replicas: 3
  selector:
    matchLabels:
      component: api
  template:
    metadata:
      labels:
        component: api
        aadpodidbinding: aks-akv-identity-binding-selector
    spec:
      containers:
        - name: api
          image: appacr.azurecr.io/app-api
          ports:
            - containerPort: 5000
          env:
            - name: PGDATABASE
              valueFrom:
                secretKeyRef:
                  name: app-prod-secrets
                  key: PGDATABASE
            - name: PGHOST
              value: postgres-cluster-ip-service-prod
            - name: PGPORT
              valueFrom:
                secretKeyRef:
                  name: app-prod-secrets
                  key: PGPORT
            - name: PGUSER
              valueFrom:
                secretKeyRef:
                  name: app-prod-secrets
                  key: PGUSER
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: app-prod-secrets
                  key: PGPASSWORD
          volumeMounts:
            - name: secrets-store01-inline
              mountPath: /mnt/secrets-store
              readOnly: true
      volumes:
        - name: secrets-store01-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: aks-akv-secret-provider
---
apiVersion: v1
kind: Service
metadata:
  name: api-cluster-ip-service-prod
  namespace: prod
spec:
  type: ClusterIP
  selector:
    component: api
  ports:
    - port: 5000
      targetPort: 5000

Then in my application settings.py:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ['PGDATABASE'],
        'USER': os.environ['PGUSER'],
        'PASSWORD': os.environ['PGPASSWORD'],
        'HOST': os.environ['PGHOST'],
        'PORT': os.environ['PGPORT'],
    }
}

Nothing in my Dockerfile refers to any of these variables, just the Django micro service code.

According to the link, one of the comments was:

current best practices advise against doing this exactly. secrets managed through environment variables in docker are easily viewed and should not be considered secure.

So I'm second guessing this approach.

Do I need to look into revising what I have here?

The suggestion in the link is to place the os.environ[] with a call to a method that pulls the credentials from a key vault... but the credentials to even access the key vault would need to be stored in secrets... so I'm not seeing how it is any different.


Note: One thing I noticed is this is the use of env: and mounting the secrets to a volume is redundant. The latter was done per the documentation on the integration, but it makes the secrets available from /mnt/secrets-store so you can do something like cat /mnt/secrets-store/PROD-PGUSER. os.environ[] isn't really necessary and the env: I don't think because you could pull the secret from that location in the Pod.

At least doing something like the following prints out the secret value:

kubectl exec -it $(kubectl get pods -l component=api -o custom-columns=:metadata.name -n prod) -n prod -- cat /mnt/secrets-store/PROD-PGUSER

Solution

  • The comment on the answer you linked was incorrect. I've left a note to explain the confusion. What you have is fine, if possibly over-built :) You're not actually gaining any security vs. just using Kubernetes Secrets directly but if you prefer the workflow around AKV then this looks fine. You might want to look at externalsecrets rather than this weird side feature of the CSI stuff? The CSI driver is more for exposing stuff as files rather than external->Secret->envvar.