I have a bash script in my ADO yaml pipeline that unzips a given set of files, some of these files are password protected. I have created a variable group and referenced it in my pipeline & defined the name of each password protected file as a key and their password as a secret value.
I would like to loop of over the target files and and for each file if there is a matching key or if its value is not empty then get the password defined under variable group. I have tried a few things but unfortunately no luck yet.
Variable group:
Key Value
test.zip **** ->(testValue)
Pipeline:
- task: Bash@3
inputs:
targetType: 'inline'
script: |
file_dir="some_dir"
if [ -d "$file_dir" ]; then
echo "Listing files in $file_dir:"
for file in "$file_dir"/*; do
if [ -f "$file" ]; then
build_var="$""("$file")"
echo $build_var -> # prints $(test.zip)
# Goal
unzip -P "$(build_var)" test.zip -d .
fi
done
fi
Since the pipeline processes lots of different files everyday I'm trying to find a way to not have to hardcode things as much as possible.
Is this possible?
Based on your description, you need to get the secret variable value in variable group based on the file name variable.
The secret variable will not be set in the Pipeline environment variable, so when you use the nested variable("$""("$file")"
) in bash script, it will not read the correct secret variable.
Since you doesn't want to hardcode the files in Pipeline, I afraid that you are not able to achieve the requirement in the single pipeline.
To meet your requirement, I suggest that you can split the pipelines into two. One is used to collect the file names(save to array) and trigger the other pipeline with the required array. The other pipeline is used to loop the file names and get the related secret password in variable group.
Here is an example:
variable Group:
file1.zip ***
file2.zip ***
Pipeline One:
pool:
vmImage: ubuntu-latest
steps:
- task: Bash@3
inputs:
targetType: 'inline'
script: |
file_dir="file_pat"
file_array=()
if [ -d "$file_dir" ]; then
echo "Listing files in $file_dir:"
for file in "$file_dir"/*; do
file_array+=$(basename $file,)
done
fi
echo $file_array
az pipelines run --id {PipelintwoID} --organization $(System.CollectionUri) --project {YourProjectname} --parameters filenames="[$file_array]"
env:
AZURE_DEVOPS_EXT_PAT: $(PAT)
Pipeline Two: We can use format: $(${{filename}})
to get the secret variable in Variable Group
parameters:
- name: filenames
type: object
default: []
variables:
- group: filePass
steps:
- ${{ each filename in parameters.filenames }}:
- bash: |
echo ${{filename}}
if [ -z "$(${{filename}})" ]
then
#variablevalue is null
unzip test.zip -d .
else
#variablevalue is not null
unzip -P "$(${{filename}})" test.zip -d .
fi
Result: It will loop all file names in the array and run related commands in different tasks.
Here is the doc related to Azure DevOps CLI trigger Pipeline: az pipelines run