kubernetesterraformazure-aksterraform-provider-azure

Create kubernetes secret for docker registry - Terraform


Using kubectl we can create docker registry authentication secret as follows

kubectl create secret docker-registry regsecret \
--docker-server=docker.example.com \
--docker-username=kube \
--docker-password=PW_STRING \
--docker-email=my@email.com \

How do i create this secret using terraform, i saw this link, it has data, in the flow of terraform the kubernetes instance is being created in azure and i get the data required from there and i created something like below

resource "kubernetes_secret" "docker-registry" {
  metadata {
    name = "registry-credentials"
  }

  data = {
    docker-server = data.azurerm_container_registry.docker_registry_data.login_server
    docker-username = data.azurerm_container_registry.docker_registry_data.admin_username
    docker-password = data.azurerm_container_registry.docker_registry_data.admin_password
  }


}

It seems that it is wrong as the images are not being pulled. What am i missing here.


Solution

  • I would suggest creating a azurerm_role_assignement to give aks access to the acr:

    resource "azurerm_role_assignment" "aks_sp_acr" {
      scope                = azurerm_container_registry.acr.id
      role_definition_name = "AcrPull"
      principal_id         = var.service_principal_obj_id
    
      depends_on = [
        azurerm_kubernetes_cluster.aks,
        azurerm_container_registry.acr
      ]
    }
    

    Update

    You can create the service principal in the azure portal or with az cli and use client_id, client_secret and object-id in terraform.

    Get Client_id and Object_id by running az ad sp list --filter "displayName eq '<name>'". The secret has to be created in the Certificates & secrets tab of the service principal. See this guide: https://pixelrobots.co.uk/2018/11/first-look-at-terraform-and-the-azure-cloud-shell/

    Just set all three as variable, eg for obj_id:

    variable "service_principal_obj_id" {
       default = "<object-id>"
    }
    

    Now use the credentials with aks:

    resource "azurerm_kubernetes_cluster" "aks" {
    
      ...
    
      service_principal {
        client_id     = var.service_principal_app_id
        client_secret = var.service_principal_password
      }
    
      ...
    
    }
    

    And set the object id in the acr as described above.

    Alternative

    You can create the service principal with terraform (only works if you have the necessary permissions). https://www.terraform.io/docs/providers/azuread/r/service_principal.html combined with a random_password resource:

    resource "azuread_application" "aks_sp" {
      name                       = "somename"
      available_to_other_tenants = false
      oauth2_allow_implicit_flow = false
    }
    
    resource "azuread_service_principal" "aks_sp" {
      application_id = azuread_application.aks_sp.application_id
    
      depends_on = [
        azuread_application.aks_sp
      ]
    }
    
    resource "azuread_service_principal_password" "aks_sp_pwd" {
      service_principal_id = azuread_service_principal.aks_sp.id
      value                = random_password.aks_sp_pwd.result
      end_date             = "2099-01-01T01:02:03Z"
    
      depends_on = [
        azuread_service_principal.aks_sp
      ]
    }
    

    You need to assign the role "Conributer" to the sp and can use it directly in aks / acr.

    resource "azurerm_role_assignment" "aks_sp_role_assignment" {
      scope                = var.subscription_id
      role_definition_name = "Contributor"
      principal_id         = azuread_service_principal.aks_sp.id
    
      depends_on = [
        azuread_service_principal_password.aks_sp_pwd
      ]
    }
    

    Use them with aks:

    resource "azurerm_kubernetes_cluster" "aks" {
    
      ...
    
      service_principal {
        client_id     = azuread_service_principal.aks_sp.app_id
        client_secret = azuread_service_principal_password.aks_sp_pwd.value
      }
    
      ...
    
    }
    

    and the role assignment:

    resource "azurerm_role_assignment" "aks_sp_acr" {
      scope                = azurerm_container_registry.acr.id
      role_definition_name = "AcrPull"
      principal_id         = azuread_service_principal.aks_sp.object_id
    
      depends_on = [
        azurerm_kubernetes_cluster.aks,
        azurerm_container_registry.acr
      ]
    }
    

    Update secret example

    resource "random_password" "aks_sp_pwd" {
      length = 32
      special = true
    }