kubernetesgitlabibm-cloudgitlab-ci

How to authenticate and access Kubernetes cluster for devops pipeline?


Normally you'd do ibmcloud loginibmcloud ks cluster-config mycluster ⇒ copy and paste the export KUBECONFIG= and then you can run your kubectl commands.

But if this were being done for some automated devops pipeline outside of IBM Cloud, what is the method for getting authenticating and getting access to the cluster?


Solution

  • You should not copy your kubeconfig to the pipeline. Instead you can create a service account with permissions to a particular namespace and then use its credentials to access the cluster.

    What I do is create a service account and role binding like this:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: gitlab-tez-dev # account name
      namespace: tez-dev #namespace
    
    ---
    kind: Role
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: tez-dev-full-access #role
      namespace: tez-dev
    rules:
      - apiGroups: ["", "extensions", "apps"]
        resources: ["deployments", "replicasets", "pods", "services"] #resources to which permissions are granted
        verbs: ["*"] # what actions are allowed
    ---
    kind: RoleBinding
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: tez-dev-view
      namespace: tez-dev
    subjects:
      - kind: ServiceAccount
        name: gitlab-tez-dev
        namespace: tez-dev
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: tez-dev-full-access
    

    Then you can get the token for the service account using:

    kubectl describe secrets -n <namespace> gitlab-tez-dev-token-<value>
    

    The output:

    Name:         gitlab-tez-dev-token-lmlwj
    Namespace:    tez-dev
    Labels:       <none>
    Annotations:  kubernetes.io/service-account.name: gitlab-tez-dev
                  kubernetes.io/service-account.uid: 5f0dae02-7b9c-11e9-a222-0a92bd3a916a
    
    Type:  kubernetes.io/service-account-token
    
    Data
    ====
    ca.crt:     1042 bytes
    namespace:  7 bytes
    token: <TOKEN>
    

    In the above command, namespace is the namespace in which you created the account and the value is the unique value which you will see when you do

    kubectl get secret -n <namespace>
    

    Copy the token to your pipeline environment variables or configuration and then you can access it in the pipeline. For example, in gitlab I do (only the part that is relevant here):

    k8s-deploy-stage:
      stage: deploy
      image: lwolf/kubectl_deployer:latest
      services:
        - docker:dind
      only:
        refs:
          - dev
      script:
         ######## CREATE THE KUBECFG ##########
        - kubectl config set-cluster ${K8S_CLUSTER_NAME} --server=${K8S_URL}
        - kubectl config set-credentials gitlab-tez-dev --token=${TOKEN}
        - kubectl config set-context tez-dev-context --cluster=${K8S_CLUSTER_NAME} --user=gitlab-tez-dev --namespace=tez-dev
        - kubectl config use-context tez-dev-context
        ####### NOW COMMANDS WILL BE EXECUTED AS THE SERVICE ACCOUNT #########
        - kubectl apply -f deployment.yml
        - kubectl apply -f service.yml
        - kubectl rollout status -f deployment.yml