I’m trying to understand the best practices around maintaining secrets in key vaults:
Question 1: Should we give infrastructure access to Azure Key Vault and have the team member access to maintain the secrets?
Question 2: Second options is to have a script that would read the values from ADO variable group and then the script would add/update them into the key vault.
I was trying to implement the second option so that we can have all the secrets managed from a single place and this is what happens.
The plan was to have those variables that are supposed to be in KeyVault start with the static string KeyVault_
and then read those using the az pipeline variable group show -- group-id $GROUP_ID
but as expected the value is empty for secrets values.
How can I achieve a similar approach and update the key vault secrets from ADO secrets.
$variable_group.variables.GetEnumerator() | ForEach-Object {
if ($_.Key.StartsWith("KeyVault_")) {
$SecretName = $_.Key -replace "KeyVault_", ""
$SecretValue = $_.Value.value
if ($SecretValue) {
Write-Output "Creating secret for $SecretName."
az keyvault secret set --vault-name $KeyVaultName --name $SecretName --value $SecretValue --output none
Write-Output "Added secret '$SecretName' to Key Vault '$KeyVaultName'."
} else {
Write-Warning "No value found for variable '$($_.Key)'."
}
}
}
How can I get the value for $SecretValue
assigned properly, I've tried $($_.Key)
but with no luck!
In the Azure DevOps project where the variable group is in, you can set up 2 YAML pipelines (ListVars and UpdateSecrets) like as below:
The PowerShell script "scripts/list-variabe-names.ps1
".
param (
# Name of Azure DevOps organization.
[Alias("org", "o")]
[string] $organization,
# Name of project where the variable group and pipelines are in.
[Alias("proj", "p")]
[string] $project,
# ID of variable group.
[Alias("gid")]
[string] $groupId,
# ID of the pipeline (UpdateSecrets) that needs to be triggered by this script.
[Alias("pid")]
[string] $PipelineId,
# Name of Azure Key Vault.
[Alias("kv")]
[string] $keyVault
)
$response = (az pipelines variable-group variable list --org "https://dev.azure.com/$organization" -p "$project" --id $groupId | ConvertFrom-Json)
$varNames = $response | Get-Member | Where {$_. MemberType -like 'NoteProperty'} | Select-Object -Property name
$varNamesList = "["
ForEach ($varName in $varNames.Name)
{
$varNamesList += "${varName},"
}
$varNamesList = $varNamesList -replace ".{1}$"
$varNamesList += "]"
az pipelines run --org "https://dev.azure.com/$organization" -p "$project" --id $PipelineId --parameters variableNames="$varNamesList" KeyVaultName="$keyVault"
The first pipeline (ListVars).
# list-variable-group-variables.yml
trigger: none
# When manually triggering this pipeline,
# you can enter new values to override the default values of parameters.
parameters:
- name: organizationName
type: string
default: 'myOrg'
- name: projectName
type: string
default: 'myProj'
- name: groupId
type: number
default: 5
- name: PipelineId
type: number
default: 83
- name: KeyVaultName
type: string
default: 'myKV'
steps:
- task: PowerShell@2
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
inputs:
pwsh: true
filePath: 'scripts/list-variabe-names.ps1'
arguments: >
-o "${{ parameters.organizationName }}"
-p "${{ parameters.projectName }}"
-gid ${{ parameters.groupId }}
-pid ${{ parameters.PipelineId }}
-kv "${{ parameters.KeyVaultName }}"
The second pipeline (UpdateSecrets).
# update-secrets.yml
trigger: none
parameters:
- name: variableNames
type: object
default: []
- name: KeyVaultName
type: string
default: ''
variables:
- group: myVarGroup
steps:
- ${{ each variableName in parameters.variableNames }}:
- task: AzureCLI@2
displayName: 'Set KeyVault secret - ${{ variableName }}'
inputs:
azureSubscription: 'myArmConnection'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
Write-Host "Setting Azure Key Vault secret - ${{ variableName }}"
az keyvault secret set --vault-name ${{ parameters.KeyVaultName }} --name ${{ variableName }} --value $(${{ variableName }})
On the Security
hub of Pipelines, ensure the build identities "Project Collection Build Service ({Organization-Name})
" and "{Project-Name} Build Service ({Organization-Name})
" have the following permissions set to "Allow
".
Queue builds
Edit queue build configuration
With above configurations, when you manually trigger the first pipeline (ListVars) with your values of parameters, the pipeline will execute the PowerShell script "scripts/list-variabe-names.ps1
" to do the following things.
az pipelines variable-group variable list
" to list all variables from the variable group. And pick up the variable names into a list.az pipelines run
" to trigger the second pipeline (UpdateSecrets) and pass the required parameter values to it.When the second pipeline is triggered, it receives the parameter values passed from the first pipeline, and then use the values to run the Azure CLI "az keyvault secret set
" to set or update secret with the new secret value into Azure Key Vault.