Maintaining secrets in Azure Key Vault so that we can limit individual access

I’m trying to understand the best practices around maintaining secrets in key vaults:

• Question 1: Should we give infrastructure access to Azure Key Vault and have the team member access to maintain the secrets?

• Question 2: Second options is to have a script that would read the values from ADO variable group and then the script would add/update them into the key vault.

I was trying to implement the second option so that we can have all the secrets managed from a single place and this is what happens.

The plan was to have those variables that are supposed to be in KeyVault start with the static string KeyVault_ and then read those using the az pipeline variable group show -- group-id $GROUP_ID but as expected the value is empty for secrets values.

How can I achieve a similar approach and update the key vault secrets from ADO secrets.

$variable_group.variables.GetEnumerator() | ForEach-Object { 
    if ($_.Key.StartsWith("KeyVault_")) {
        $SecretName = $_.Key -replace "KeyVault_", ""
        $SecretValue = $_.Value.value

        if ($SecretValue) {
            Write-Output "Creating secret for $SecretName."
            az keyvault secret set --vault-name $KeyVaultName --name $SecretName --value $SecretValue --output none
            Write-Output "Added secret '$SecretName' to Key Vault '$KeyVaultName'."
        } else {
            Write-Warning "No value found for variable '$($_.Key)'."
        }
    }
}

How can I get the value for $SecretValue assigned properly, I've tried $($_.Key) but with no luck!

In the Azure DevOps project where the variable group is in, you can set up 2 YAML pipelines (ListVars and UpdateSecrets) like as below:

1. The PowerShell script "scripts/list-variabe-names.ps1".

   param (
       # Name of Azure DevOps organization.
       [Alias("org", "o")]
       [string] $organization,
   
       # Name of project where the variable group and pipelines are in.
       [Alias("proj", "p")]
       [string] $project,
   
       # ID of variable group.
       [Alias("gid")]
       [string] $groupId,
   
       # ID of the pipeline (UpdateSecrets) that needs to be triggered by this script.
       [Alias("pid")]
       [string] $PipelineId,
   
       # Name of Azure Key Vault.
       [Alias("kv")]
       [string] $keyVault
   )
   
   $response = (az pipelines variable-group variable list --org "https://dev.azure.com/$organization" -p "$project" --id $groupId | ConvertFrom-Json)
   $varNames = $response | Get-Member | Where {$_. MemberType -like 'NoteProperty'} | Select-Object -Property name
   
   $varNamesList = "["
   ForEach ($varName in $varNames.Name)
   {
       $varNamesList += "${varName},"
   }
   
   $varNamesList = $varNamesList -replace ".{1}$"
   $varNamesList += "]"
   
   az pipelines run --org "https://dev.azure.com/$organization" -p "$project" --id $PipelineId --parameters variableNames="$varNamesList" KeyVaultName="$keyVault"
   

2. The first pipeline (ListVars).

   # list-variable-group-variables.yml
   
   trigger: none
   
   # When manually triggering this pipeline,
   # you can enter new values to override the default values of parameters.
   parameters:
   - name: organizationName
     type: string
     default: 'myOrg'
   
   - name: projectName
     type: string
     default: 'myProj'
   
   - name: groupId
     type: number
     default: 5
   
   - name: PipelineId
     type: number
     default: 83
   
   - name: KeyVaultName
     type: string
     default: 'myKV'
   
   steps:
   - task: PowerShell@2
     env:
       AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
     inputs:
       pwsh: true
       filePath: 'scripts/list-variabe-names.ps1'
       arguments: >
         -o "${{ parameters.organizationName }}"
         -p "${{ parameters.projectName }}"
         -gid ${{ parameters.groupId }}
         -pid ${{ parameters.PipelineId }}
         -kv "${{ parameters.KeyVaultName }}"
   

   

3. The second pipeline (UpdateSecrets).

   # update-secrets.yml
   
   trigger: none
   
   parameters:
   - name: variableNames
     type: object
     default: []
   
   - name: KeyVaultName
     type: string
     default: ''
   
   variables:
   - group: myVarGroup
   
   steps:
   - ${{ each variableName in parameters.variableNames }}:  
     - task: AzureCLI@2
       displayName: 'Set KeyVault secret - ${{ variableName }}'
       inputs:
         azureSubscription: 'myArmConnection'
         scriptType: 'pscore'
         scriptLocation: 'inlineScript'
         inlineScript: |
           Write-Host "Setting Azure Key Vault secret - ${{ variableName }}"
           az keyvault secret set --vault-name ${{ parameters.KeyVaultName }} --name ${{ variableName }} --value $(${{ variableName }})
   

4. On the Security hub of Pipelines, ensure the build identities "Project Collection Build Service ({Organization-Name})" and "{Project-Name} Build Service ({Organization-Name})" have the following permissions set to "Allow".

   • Queue builds
   • Edit queue build configuration

   

With above configurations, when you manually trigger the first pipeline (ListVars) with your values of parameters, the pipeline will execute the PowerShell script "scripts/list-variabe-names.ps1" to do the following things.

• Call the Azure DevOps CLI "az pipelines variable-group variable list" to list all variables from the variable group. And pick up the variable names into a list.
• Call CLI "az pipelines run" to trigger the second pipeline (UpdateSecrets) and pass the required parameter values to it.

When the second pipeline is triggered, it receives the parameter values passed from the first pipeline, and then use the values to run the Azure CLI "az keyvault secret set" to set or update secret with the new secret value into Azure Key Vault.