I have defined a main playbook, which calls another preparation playbook via import_playbook.
The preparation playbook returns credentials for an S3 repository, var is called "service_secrets" at the moment, which is a dict (see below). This dict needs to be appended into the last item of product.dependencies. The indicator to pick the specific listitem could be just "Source=S3".
---
- name: 'Preparation'
ansible.builtin.import_playbook: 'playbook_preparation.yml'
vars:
product:
name: 'mssql'
dependencies:
- Name: 'SqlServer'
Source: 'psgallery'
Type: 'psmodule'
- Name: 'SqlServerDsc'
Source: 'psgallery'
Type: 'dscmodule'
- Name: 'AWSPowershell.NetCore'
Source: 'psgallery'
Type: 'psmodule'
- Name: 'ComputerManagementDsc'
Source: 'psgallery'
Type: 'dscmodule'
- Name: >-
{{
["mssql", ci.attributes.input_parameters.mssql.version,
ci.attributes.input_parameters.mssql.edition,
ci.attributes.input_parameters.mssql.language]
| join("_")
}}
Source: 'S3'
Type: 'iso'
Bucket: 'repository'
The new fact "service_secrets" looks something like this:
"service_secrets": {
"Access_key_id": "abc",
"Access_key_secret": "defghi",
"Bucket": "repository",
"Endpoint": "https://s3.abc.com"
}
I tried to fiddle around with combine(), but i could not get the service_secrets into the last listitem.
- name: DEBUG
debug:
msg: '{{ product | combine({"dependencies": [service_secrets]}, list_merge="append") }}'
loop: '{{ product.dependencies }}'
when: 'item.Source == "S3"'
This will result in:
ok: [localhost] => (item={'Name': 'mssql_2022_standard_en', 'Source': 'S3', 'Type': 'iso', 'Bucket': 'repository'}) => {
"msg": {
"dependencies": [
{
"Name": "SqlServer",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "SqlServerDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Name": "AWSPowershell.NetCore",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "ComputerManagementDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Bucket": "repository",
"Name": "mssql_2022_standard_en",
"Source": "S3",
"Type": "iso"
},
{
"Access_key_id": "abc",
"Access_key_secret": "defgh",
"Bucket": "repository",
"Endpoint": "https://s3.abc.com"
}
],
"name": "mssql"
}
}
To make it more clear: The last two listitems of product.dependencies should be one item.
There are several ways to achieve this, I would like to show you 2 different ones, as both have advantages and disadvantages.
Your initial data
vars:
dependencies:
- Name: 'SqlServer'
Source: 'psgallery'
Type: 'psmodule'
- Name: 'SqlServerDsc'
Source: 'psgallery'
Type: 'dscmodule'
- Name: 'AWSPowershell.NetCore'
Source: 'psgallery'
Type: 'psmodule'
- Name: 'ComputerManagementDsc'
Source: 'psgallery'
Type: 'dscmodule'
- Name: mssql_2022_standard_en
Source: 'S3'
Type: 'iso'
Bucket: 'repository'
- Name: 'AnotherDependency_notFromS3'
Source: 'psgallery'
Type: 'dscmodule'
service_secrets:
Access_key_id: abc
Access_key_secret: defghi
Bucket: repository
Endpoint: https://s3.abc.com
Can be implemented in a single task without having to prepare the data in a separate task. First, your dependencies list is split into elements Source == 'S3' yes or no.
The splitting works with the rejectattr() or selectattr() functions. The elements whose attribute Source == 'S3' are either rejected or filtered out (selected).
For all elements whose source is S3, the service_secrets are added. The service_secrets are added using map(), whereby a combine(service_secrets) is performed for each element in the list.
The two separate lists are then appended together again.
I see this as the Ansible way to implement this. The disadvantage is that your dependencies list may then have a different sort order than at the beginning. Compare the sort order of both results. However, if the order does not matter, I would recommend this method.
Task code
- name: Variant 1 - split by Source, enrich and recombine
debug:
msg: "{{ dependencies_not_s3 + dependencies_s3 }}"
vars:
dependencies_not_s3: "{{ dependencies | rejectattr('Source', 'eq', 'S3') }}"
dependencies_s3: "{{ dependencies | selectattr('Source', 'eq', 'S3') | map('combine', service_secrets) }}"
Result
TASK [Variant 1 - split by Source, enrich and recombine] ************
ok: [localhost] => {
"msg": [
{
"Name": "SqlServer",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "SqlServerDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Name": "AWSPowershell.NetCore",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "ComputerManagementDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Name": "AnotherDependency_notFromS3",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Access_key_id": "abc",
"Access_key_secret": "defghi",
"Bucket": "repository",
"Endpoint": "https://s3.abc.com",
"Name": "mssql_2022_standard_en",
"Source": "S3",
"Type": "iso"
}
]
}
In the second variant, you iterate over the dependencies list as the initial set and create a new list using the set_fact module. Depending on the source, the new item is first enriched with the service_secrets.
An iteration step of a task is required for each entry in the dependencies list, but this ensures that the original sort order of the dependencies list is retained.
Be careful if you do this several times for different dependencies lists that you start with an empty new variable.
Task code
- name: Variant 2 - iterate over dependencies
set_fact:
extended_dependencies: "{{ (extended_dependencies | default([])) + [new_item] }}"
vars:
new_item: "{{ item | combine(service_secrets) if item.Source == 'S3' else item }}"
with_items: "{{ dependencies }}"
- name: Print result from Variant 2
debug:
var: extended_dependencies
Result
TASK [Variant 2 - iterate over dependencies] **************************************************************************
ok: [localhost] => (item={'Name': 'SqlServer', 'Source': 'psgallery', 'Type': 'psmodule'})
ok: [localhost] => (item={'Name': 'SqlServerDsc', 'Source': 'psgallery', 'Type': 'dscmodule'})
ok: [localhost] => (item={'Name': 'AWSPowershell.NetCore', 'Source': 'psgallery', 'Type': 'psmodule'})
ok: [localhost] => (item={'Name': 'ComputerManagementDsc', 'Source': 'psgallery', 'Type': 'dscmodule'})
ok: [localhost] => (item={'Name': 'mssql_2022_standard_en', 'Source': 'S3', 'Type': 'iso', 'Bucket': 'repository'})
ok: [localhost] => (item={'Name': 'AnotherDependency_notFromS3', 'Source': 'psgallery', 'Type': 'dscmodule'})
TASK [Print result from Variant 2] ************************************************************************************
ok: [localhost] => {
"extended_dependencies": [
{
"Name": "SqlServer",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "SqlServerDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Name": "AWSPowershell.NetCore",
"Source": "psgallery",
"Type": "psmodule"
},
{
"Name": "ComputerManagementDsc",
"Source": "psgallery",
"Type": "dscmodule"
},
{
"Access_key_id": "abc",
"Access_key_secret": "defghi",
"Bucket": "repository",
"Endpoint": "https://s3.abc.com",
"Name": "mssql_2022_standard_en",
"Source": "S3",
"Type": "iso"
},
{
"Name": "AnotherDependency_notFromS3",
"Source": "psgallery",
"Type": "dscmodule"
}
]
}