kuberneteskubernetes-apiserver

Kubernetes: set environment variables from file?


My Kubernetes deployment has an initContainer which fetches a token from a URL. My app container (3rd party) then needs that token as an environment variable.

A possible approach would be: the initContainer creates a Kubernetes Secret with the token value; the app container uses the secret as an environment variable via env[].valueFrom.secretKeyRef.

Creating the Secret from the initContainer requires accessing the Kubernetes API from a Pod though, which tends to be a tad cumbersome. For example, directly accessing the REST API requires granting proper permissions to the pod's service account; otherwise, creating the secret will fail with

secrets is forbidden: User \"system:serviceaccount:default:default\" 
cannot create resource \"secrets\" in API group \"\" in the namespace \"default\"

So I was wondering, isn't there any way to just write the token to a file on an emptyDir volume...something like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: my-app
spec:
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      initContainers:
        - name: fetch-auth-token
          image: curlimages/curl
          command:
            - /bin/sh
          args:
            - -c
            - |
              echo "Fetching token..."
              url=https://gist.githubusercontent.com/MaxHorstmann/a99823d5aff66fe2ad4e7a4e2a2ee96b/raw/662c19aa96695e52384337bdbd761056bb324e72/token
              curl $url > /auth-token/token
          volumeMounts:
            - mountPath: /auth-token
              name: auth-token
...
      volumes:
        - name: auth-token
          emptyDir: {}

... and then somehow use that file to populate an environment variable in the app container, similar to env[].valueFrom.secretKeyRef, along the lines of:

      containers:
        - name: my-actual-app
          image: thirdpartyappimage
          env:
            - name: token
              valueFrom:
                fileRef:
                  path: /auth-token/token
            # ^^^^ this does not exist
          volumeMounts:
            - mountPath: /auth-token
              name: auth-token

Unfortunately, there's no env[].valueFrom.fileRef.

I considered overwriting the app container's command with a shell script which loads the environment variable from the file before launching the main command; however, the container image doesn't even contain a shell.

Is there any way to set the environment variable in the app container from a file?


Solution

  • Creating the Secret from the initContainer requires accessing the Kubernetes API from a Pod though, which tends to be a tad cumbersome...

    It's not actually all that bad; you only need to add a ServiceAccount, Role, and RoleBinding to your deployment manifests.

    The ServiceAccount manifest is minimal, and you only need it if you don't want to grant permissions to the default service account in your namespace:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: secretmaker
    

    Then your Role grants access to secrets (we need create and delete permissions, and having get and list is handy for debugging):

    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      labels:
        app: env-example
      name: secretmaker
    rules:
    - apiGroups:
      - ""
      resources:
      - secrets
      verbs:
      - create
      - get
      - delete
      - list
    

    A RoleBinding connects the ServiceAccount to the Role:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      labels:
        app: env-example
      name: secretmaker
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: secretmaker
    subjects:
    - kind: ServiceAccount
      name: secretmaker
      namespace: default
    

    And with those permissions in place, the Deployment is relatively simple:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: env-example
      name: env-example
      namespace: env-example
    spec:
      selector:
        matchLabels:
          app: env-example
      template:
        metadata:
          labels:
            app: env-example
        spec:
          serviceAccountName: secretmaker
          initContainers:
          - command:
            - /bin/sh
            - -c
            - |
              echo "Fetching token..."
              url=https://gist.githubusercontent.com/MaxHorstmann/a99823d5aff66fe2ad4e7a4e2a2ee96b/raw/662c19aa96695e52384337bdbd761056bb324e72/token
              curl $url -o /tmp/authtoken
              kubectl delete secret authtoken > /dev/null 2>&1
              kubectl create secret generic authtoken --from-file=AUTH_TOKEN=/tmp/authtoken
            image: docker.io/alpine/k8s:1.25.6
            name: create-auth-token
          containers:
          - name: my-actual-app
            image: docker.io/alpine/k8s:1.25.6
            command:
            - sleep
            - inf
            envFrom:
            - secretRef:
                name: authtoken
    

    The application container here is a no-op that runs sleep inf; that gives you the opportunity to inspect the environment by running:

    kubectl exec -it deployment/env-example -- env
    

    Look for the AUTH_TOKEN variable created by our initContainer.


    All the manifests mentioned here can be found in this repository.