ansiblejinja2ansible-template

Create multiple files from the same Jinja2 Template using values from a var file

I need to create multiple files with values declared in a var file from a Jinja2 Template file (server.properties). I have tried known methods to implement the file generation but the problem is only the first variable changes. Others are not reflected into this template but the files are created perfectly. I'm attaching the necessary files with this post.

Main.yml

---
- name: Create server properties files
  hosts: localhost
  become: yes
  become_user: root
  gather_facts: true
  vars:
    t_file: server.properties
    t_dir: /path/to/save/newfiles
  tasks:
    - name: Include vars file
      include_vars:
        file: common.yml
        name: common

    - name: Creating template file - server.properties.j2
      template:
        src: "{{ t_file }}"
        dest: 'server.properties.j2'

common.yml

---
server:
  - server_no: 1
  - server_no: 2
node_id:
  - node_id: 1
  - node_id: 2
cqvoters:
  - cq_voters: 1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082
  - cq_voters: 1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082
listeners:
  - listeners: SASL_SSL://web.example.com:18080,CONTROLLER://web.example.com:8080
  - listeners: SASL_SSL://web.example.com:18081,CONTROLLER://web.example.com:8081
advertised:
  - advertised_listeners: SASL_SSL://web.example.com:18080
  - advertised_listeners: SASL_SSL://web.example.com:18081
log_dir:
  - log_dir: /tmp/server1/kraft-combined-logs
  - log_dir: /tmp/server2/kraft-combined-logs

server.properties:

node.id= "{{ node_id }}"
controller.quorum.voters= "{{ cqvoters }}"
############################# Socket Server Settings #############################
listeners= "{{ listeners }}"
advertised.listeners= "{{ advertised }}"
############################# Log Basics #############################
log.dirs= "{{ log_dir }}"

As per the above given contents, I need to create 2 server.properties files (i.e. server1.properties and server2.properties) with the values from common.yml so that, as an example, the final output of server1.properties should be:

node.id= 1
controller.quorum.voters= 1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082
Socket Server Settings #############################
listeners= SASL_SSL://web.example.com:18080,CONTROLLER://web.example.com:8080
advertised.listeners= SASL_SSL://web.example.com:18080
Log Basics #############################
log.dirs= /tmp/server1/kraft-combined-logs
Solution

  • You are making your life hard with an unadapted data structure. Here's a minimal example with a better one to achieve your overall goal. Adapt to your exact needs.

    Project Structure:

    $ tree
.
├── playbook.yml
├── templates
│   └── server.properties.j2
└── vars
    └── common.yml

2 directories, 3 files

    vars/common.yml:

    ---
properties_by_nodes:
  - node_id: 1
    cqvoters: 1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082
    listeners: SASL_SSL://web.example.com:18080,CONTROLLER://web.example.com:8080
    advertised: SASL_SSL://web.example.com:18080
    log_dir: /tmp/server1/kraft-combined-logs
  - node_id: 2
    cqvoters: 1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082
    listeners: SASL_SSL://web.example.com:18081,CONTROLLER://web.example.com:8081
    advertised: SASL_SSL://web.example.com:18081
    log_dir: /tmp/server2/kraft-combined-logs

    templates/server.properties.j2:

    node.id= "{{ node_id }}"
controller.quorum.voters= "{{ cqvoters }}"
############################# Socket Server Settings #############################
listeners= "{{ listeners }}"
advertised.listeners= "{{ advertised }}"
############################# Log Basics #############################
log.dirs= "{{ log_dir }}"

    playbook.yml:

    - name: Create different properties files from variable
  hosts: localhost
  gather_facts: false

  vars_files:
    - common.yml

  tasks:
    - name: Create a properties file for each entry in properties_by_nodes
      vars:
        node_id: "{{ item.node_id }}"
        cqvoters: "{{ item.cqvoters }}"
        listeners: "{{ item.listeners }}"
        advertised: "{{ item.advertised }}"
        log_dir: "{{ item.log_dir }}"
      ansible.builtin.template:
        src: server.properties.j2
        dest: "/tmp/server{{ node_id }}.properties"
      loop: "{{ properties_by_nodes }}"

    Running the playbook gives (note that running a second time with existing files with the correct content will report "ok" instead of "changed")

    $ ansible-playbook playbook.yml 

PLAY [Create different properties files from variable] ***********************

TASK [Create a properties file for each entry in properties_by_nodes] ********
changed: [localhost] => (item={'node_id': 1, 'cqvoters': '1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082', 'listeners': 'SASL_SSL://web.example.com:18080,CONTROLLER://web.example.com:8080', 'advertised': 'SASL_SSL://web.example.com:18080', 'log_dir': '/tmp/server1/kraft-combined-logs'})
changed: [localhost] => (item={'node_id': 2, 'cqvoters': '1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082', 'listeners': 'SASL_SSL://web.example.com:18081,CONTROLLER://web.example.com:8081', 'advertised': 'SASL_SSL://web.example.com:18081', 'log_dir': '/tmp/server2/kraft-combined-logs'})

PLAY RECAP *******************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

    And we can now check the files have been created with the correct content:

    $ for f in /tmp/server*.properties; \
do echo -e "content of $f\n----"; cat $f; echo; done

content of /tmp/server1.properties
----
node.id= "1"
controller.quorum.voters= "1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082"
############################# Socket Server Settings #############################
listeners= "SASL_SSL://web.example.com:18080,CONTROLLER://web.example.com:8080"
advertised.listeners= "SASL_SSL://web.example.com:18080"
############################# Log Basics #############################
log.dirs= "/tmp/server1/kraft-combined-logs"

content of /tmp/server2.properties
----
node.id= "2"
controller.quorum.voters= "1@web.example.com:8080,2@web.example.com:8081,3@web.example.com:8082"
############################# Socket Server Settings #############################
listeners= "SASL_SSL://web.example.com:18081,CONTROLLER://web.example.com:8081"
advertised.listeners= "SASL_SSL://web.example.com:18081"
############################# Log Basics #############################
log.dirs= "/tmp/server2/kraft-combined-logs"