listansiblejinja2f5

Ansible: Combining 2 Lists by a Specific Attribute


I have 2 lists that I'm trying to combine together, by a specific attribute (the "device" attribute). I am not instantiating the lists in my code anywhere, they are being populated by API calls so the best I have here are printouts of each list.

List1:

TASK [Print List1] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool"
            },
            {
                "device": "LTM2_Device",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool"
            },
            {
                "device": "LTM3_Device",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool"
            }
        ]
    ]
}

List2:

TASK [Print List2] ****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host1",
                "ip": "0.0.0.1",
                "port": "5555"
            },
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.2",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host1",
                "ip": "0.0.0.3",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.4",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host1",
                "ip": "0.0.0.5",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host2",
                "ip": "0.0.0.6",
                "port": "5555"
            }
        ]
    ]
}

I have 2 debug lines that print something fairly close to what I am looking for, however neither are quite what I want... and I can't seem to get the Jinja2 filter correct.

- name: Debug
    debug:
        msg: 
            - "{{ (list2 + list1) | groupby('device') | map('last') | map('combine') | list }}"
            - "{{ (list2 + list1) | groupby('device') | map('last') | list }}"

Which gives the following output:

TASK [Debug] ****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.1",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.3",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.5",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            }
        ],
        [
            [
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host1",
                    "ip": "0.0.0.1",
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host2",
                    "ip": "0.0.0.2",
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                }
            ],
            [
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host1",
                    "ip": "0.0.0.3",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host2",
                    "ip": "0.0.0.4",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                }
            ],
            [
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host1",
                    "ip": "0.0.0.5",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host2",
                    "ip": "0.0.0.6",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                }
            ]
        ]
    ]
}

The first debug line does not combine both lists for all elements of list2 (it misses the first host for each LTM), and the second line combines them as a separate element in a nested list. Is there a way to combine both lists so it would look something like the following? I know I'm probably missing something small but I can't seem to figure out what it is I'm missing. Any help would be greatly appreciated, thanks!

Desired output (I'm not worried about it being a nested list as I can flatten that later):

        [
            [
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host1",
                    "ip": "0.0.0.1",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host2",
                    "ip": "0.0.0.2",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                    "port": "5555"
                }
            ],
            [
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host1",
                    "ip": "0.0.0.3",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host2",
                    "ip": "0.0.0.4",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                    "port": "5555"
                }
            ],
            [
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host1",
                    "ip": "0.0.0.5",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host2",
                    "ip": "0.0.0.6",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                    "port": "5555"
                }
            ]
        ]

Solution

  • With the help of a peer, I was able to figure it out by looping the second list, adding a selectattr filter to match my attribute against the first list, and combining that result with the second list to create a new list of results.

    Filtered Loop:

    - name: Combine lists
      set_fact:
        new_list: "{{ new_list | default([]) + [item|combine(filter_time)] }}"
      loop: "{{ list2 }}"
      vars:
        filter_time: "{{ list1 | selectattr('device', '==', item.device) | first }}"
    

    Output:

    ok: [localhost] => {
        "msg": [
            [
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host1",
                    "ip": "0.0.0.1",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool",
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host2",
                    "ip": "0.0.0.2",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host1",
                    "ip": "0.0.0.3",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host2",
                    "ip": "0.0.0.4",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host1",
                    "ip": "0.0.0.5",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host2",
                    "ip": "0.0.0.6",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool",
                    "port": "5555"
                }
            ]
        ]
    }