Multiple secrets on kubernetes

Best practice for managing 3 or 4 secrets for a single Kubernetes deployment.

We have a deployment with some secrets repeated in all namespaces, and others that are environment specific.

We are trying to decide between one secret file and changing only the ones that need to be, OR running 2+ secrets where one is foo-dev-secrets and the other is foo-universal-secrets.

We can't find any examples of what to do in these cases, more specifically how to manage these secrets, we know you can only have "one secret per volume", but in all honesty we aren't sure what that means.

Feel free to respond as if we are dumb children. ^_^

Solution

running 2+ secrets where one is foo-dev-secrets and the other is foo-universal-secrets

As a for-your-consideration, a well-constructed RBAC policy would ensure that only accounts with the correct permissions would be able to read secrets if they are decomposed, which would (of course) be much harder if they were all lumped into one "all-the-secrets" bucket

we know you can only have "one secret per volume", but in all honesty we aren't sure what that means.

Then you're in good company, because I don't know what that means, either :-D If you mean there are global secrets, but the dev secrets are named the same but more specific, I think docker -v will tolerate that:

containers:
- name: foo
  volumeMounts:
  - name: global-secrets
    mountPoint: /run/secrets/global
    readOnly: true
  - name: foo-secret-override
    mountPoint: /run/secrets/global/no-really
    readOnly: true

... with the disadvantage being that your volumes: and volumeMounts: will become quite chatty

That said, I would bet the more general, and less mind-bend-y solution is to mount them as peers and then do the application equivalent of find /run/secrets/ -not -type d to slurp them all up:

  volumeMounts:
  - name: global-secrets
    mountPoint: /run/secrets/0-global
    readOnly: true
  - name: foo-secrets
    mountPoint: /run/secrets/1-foo
    readOnly: true

Or, if possible, have the application read the path from an environment variable or ConfigMap-ed situation, meaning one could project them as peers (still) but point-out to the application which of the two values it should use.

Of course, the devil's in the details, so feel free to chime in if you are able to share more specifics about the hurdles you are encountering