I'm attempting to automate VM deployment on XCP-ng (version 20.04.01) using Ansible and the xenserver_guest module. The guest OS is AlmaLinux 9.
ansible --version
ansible 2.9.27
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/automation/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Jun 20 2023, 11:36:40) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
My objective is to:
Problem:
The VM is successfully created, but the static IP is not assigned to the guest OS (AlmaLinux 9) after logging in.
Additional Information:
xenserver_guest module version supports setting static IPs.Context:
We're in the process of automating virtual machine (VM) deployments on our XCP-ng virtualization platform. Our current infrastructure heavily relies on manual processes, and we're aiming to streamline VM provisioning and configuration using Ansible.
This automation effort is crucial for:
Enhancing Efficiency: Automating VM deployment reduces manual intervention, saving time and effort for our IT team. Ensuring Consistency: Ansible playbooks guarantee consistent configurations across all deployed VMs, minimizing human error and configuration drift. Scalability: Automation empowers us to handle larger deployments efficiently and cater to growing infrastructure demands. We acknowledge the existence of alternative tools for VM automation. However, our current environment leverages XCP-ng and Ansible, and this project serves as a stepping stone towards a more comprehensive automation strategy. Successfully incorporating static IP assignment during VM deployment using the xenserver_guest module is a critical step in this direction.
I've attempted to configure the static IP within the networks section of the xenserver_guest module:
- name: "Create the VM with network configuration"
xenserver_guest:
validate_certs: false
name: "{{ vm_name }}"
name_desc: "Created_by: Ansible"
template: "{{ template_name }}"
# ... other VM configuration options
networks:
- name: "Pool-wide network associated with eth0"
type: static
ip: 172.16.29.139
netmask: 255.255.255.0
gateway: 172.16.29.201
# ... configuration for other network interfaces (if applicable)
state: poweredon
Also I got this output:
xcp-ng-host1 | CHANGED => {
"changed": true,
"instance": {
"cdrom": {
"iso_name": "AlmaLinux-9.2-x86_64-dvd.iso",
"type": "iso"
},
"customization_agent": "custom",
"disks": [
{
"name": "Alma_Linux_Onmobile 0",
"name_desc": "Created by template provisioner",
"os_device": "xvda",
"size": 42949672960,
"sr": "host2",
"sr_uuid": "80fd9e14-a4dc-fa21-c276-a1799d4b6bcb",
"vbd_userdevice": "0"
}
],
"domid": "322",
"folder": "",
"hardware": {
"memory_mb": 4096,
"num_cpu_cores_per_socket": 1,
"num_cpus": 4
},
"home_server": "",
"is_template": false,
"name": "my_first_vm",
"name_desc": "Created_by: Ansible",
"networks": [
{
"gateway": "172.16.29.201",
"gateway6": "",
"ip": "",
"ip6": [],
"mac": "12:ed:b7:4b:f4:33",
"mtu": "1500",
"name": "Pool-wide network associated with eth0",
"netmask": "255.255.255.0",
"prefix": "24",
"prefix6": "",
"vif_device": "0"
},
{
"gateway": "",
"gateway6": "",
"ip": "",
"ip6": [],
"mac": "36:50:3b:bd:ad:2e",
"mtu": "1500",
"name": "Pool-wide network associated with eth1",
"netmask": "",
"prefix": "",
"prefix6": "",
"vif_device": "1"
},
{
"gateway": "",
"gateway6": "",
"ip": "",
"ip6": [],
"mac": "e2:1d:ef:3a:60:1f",
"mtu": "1500",
"name": "Pool-wide network associated with eth2",
"netmask": "",
"prefix": "",
"prefix6": "",
"vif_device": "2"
},
{
"gateway": "",
"gateway6": "",
"ip": "",
"ip6": [],
"mac": "be:a2:19:ff:85:af",
"mtu": "1500",
"name": "Pool-wide network associated with eth3",
"netmask": "",
"prefix": "",
"prefix6": "",
"vif_device": "3"
}
],
"other_config": {
"auto_poweron": "false",
"base_template_name": "AlmaLinux 8",
"import_task": "OpaqueRef:67e31e5d-7ed5-4a6a-9cc2-e665f9ed69d3",
"install-methods": "cdrom,nfs,http,ftp",
"instant": "true",
"linux_template": "true",
"mac_seed": "c4366f2c-2707-1016-7c61-8c0a9773cdcd"
},
"platform": {
"acpi": "1",
"apic": "true",
"device-model": "qemu-upstream-compat",
"device_id": "0001",
"hpet": "true",
"nx": "true",
"pae": "true",
"secureboot": "false",
"timeoffset": "-1",
"vga": "std",
"videoram": "8",
"viridian": "false"
},
"state": "poweredoff",
"uuid": "f5deb35e-7f86-a75f-ab7b-74ff1bcb2345",
"xenstore_data": {
"vm-data": "",
"vm-data/mmio-hole-size": "268435456",
"vm-data/networks": "",
"vm-data/networks/0": "",
"vm-data/networks/0/gateway": "172.16.29.201",
"vm-data/networks/0/ip": "172.16.29.139",
"vm-data/networks/0/mac": "12:ed:b7:4b:f4:33",
"vm-data/networks/0/name": "Pool-wide network associated with eth0",
"vm-data/networks/0/netmask": "255.255.255.0",
"vm-data/networks/0/prefix": "24",
"vm-data/networks/0/type": "static"
}
}
}
I expected the guest OS (AlmaLinux 9) to receive the static IP configuration (172.16.29.139) after the VM deployment. However, upon logging in, the IP is not set.
Question:
xenserver_guest module from setting the static IP within the guest OS?Bojan here, the author of xenserver_* Ansible modules. Thanks for taking the interest in XenServer/XCP-ng and Ansible. I appologise for a very late reply but your question caught my attention just now and I believe it deserves a proper answer. Also thanks to Alexander for noticing documentation note and stearing you in the right direction.
TL;DR
Network parameter configuration like IP address, netmask and gateway on guest OS level is only supported for Windows based systems. Linux and other *nix based systems are not supported. To help users deal with this XenServer/XCP-ng limitation (or lack of support), xenserver_guest Ansible module exports network parameters to XenStore. User created custom scripts preinstalled into VM templete and run during VM boot can read these network parameters from XenStore and configure network interfaces accordingly (no need for ISOs or other hacks). For this to work correctly, prerequisites are:
xenstore read command to read all the network parameters from XenStore and generate /etc/sysconfig/network-scripts/ifcfg-* files or use nmcli to configure network interfaces.systemd unit that starts custom script on first boot.Full answer
Since network parameters like IP address, netmask, gateway etc. are configured on an OS level, either through /etc/sysconfig/network-scripts/ifcfg-*, nmcli, /etc/network/interfaces, netplan or other mechanisms, xenserver_guest Ansible module would require direct access to the OS to be able to configure network parameters. Should I note that no such direct access method exists? There is no way for a XenServer/XCP-ng host to directly influence or change OS configuration (effectively file system content) on the fly, not because it is technicaly impossible but because it is a serious security issue. What is possible and acceptable by security standards and what XenServer/XCP-ng offers as a middle ground is a shared key value store (or bus) called XenStore.
You can immagine XenStore as a common bus that stands between a host (hypervisor) and a guest (VM). Both the host and the guest can read and write some vales to this bus but cannot directly trigger any action on either side:
host ⟵ read/write ⟶ XenStore ⟵ read/write ⟶ guest
Now we can see that a host and a guest have a way to exchange information. So, can we make the host write some info to the bus and make the guest read it and do something with the info? Sure, but here is a catch. Guest would have to have some kind of daemon running in the background, listening for the stuff on the bus, consuming the info, and acting on that info when required. We will call this daemon "an agent".
xenserver_guest Ansible module solves the first part of the problem. In other words, it can write network parameters to the XenStore (via host). For the guest side of things, unfortunately, there is no such agent that can read the network parameters from the XenStore and configure network interfaces on the OS accordingly. That part is missing. To be more precise, officialy, Citrix never implemented such an agent for Linux based operating systems. Citrix did implement such an agent for Windows but documents mentioning this are long gone and functionality is considered obscure and experimental. No alternative (non Citrix) agent exists either, to my knowledge, that supports network parameter configuration. There was some OpenStack agent developed by RackSpace but is outdated, unmaintained and long forgoten.
So what can we do about it? We can improvise our agent. For that, we need:
This is a general requirement by XenServer/XCP-ng. All VMs should have guest tools installed but if this is not the case for you, please follow the instructions on this page to install:
https://xcp-ng.org/docs/guests.html#linux
I recommend the "Install from the guest tools ISO" approach. By installing guest tools we acquire a bunch of xenstore tools that will be usefull to us.
xenstore tools and implement a bash scriptExperiment with xenstore ls and xenstore read CLI tools inside a guest to find out and read network parameters stored in XenStore by xenserver_guest Ansible module. With this knowledge, you can easily whip up a bash script that reads all the necesarry info and configure network interfaces via ifcfg files, nmcli or by other means. man xenstore should be of great help.
Note that xenstore tools can be run in two ways:
xenstore read (read is a subcommand of xenstore)xensore-read (separate tool)Both are valid and you will see both in the documents found on the internet.
Here are some examples:
[root@noname ~]# xenstore ls vm-data
networks = ""
0 = ""
gateway = "192.168.0.1"
netmask = "255.255.255.0"
prefix = "24"
ip = "192.168.0.2"
type = "static"
mac = "76:01:e3:a6:b3:00"
name = "Host internal management network"
[root@noname ~]# xenstore read vm-data/networks/0/ip
192.168.0.2
[root@noname ~]# xenstore read vm-data/networks/0/netmask
255.255.255.0
[root@noname ~]# xenstore read vm-data/networks/0/gateway
192.168.0.1
Now it should be easy to make a bash script /usr/local/bin/setupmynetwork using these values:
#!/bin/bash
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
ONBOOT="yes"
TYPE="Ethernet"
BOOTPROTO="static"
IPADDR="$(xenstore read vm-data/networks/0/ip)"
NETMASK="$(xenstore read vm-data/networks/0/netmask)"
GATEWAY="$(xenstore read vm-data/networks/0/gateway)"
EOF
nmcli con load /etc/sysconfig/network-scripts/ifcfg-eth0
nmcli con up "System eth0"
The easiest way to do it is to put the script in rc.local:
[root@noname ~]# echo "/usr/local/bin/setupmynetwork" >> /etc/rc.d/rc.local
[root@noname ~]# chmod a+x /etc/rc.d/rc.local
Alternatively, a dedicated systemd unit can be implemented or even a cron job can be used.
And we are done. Now we have a trivial agent. We can shutdown the VM and convert it to template and use it to provision new VMs.
I personaly have not implemented a more robust agent but relied on DHCP for network parameter assignment. So that is another alternative approach worth mentioning.
A more proper solution would be to use cloud-init as an agent. Unfortunately, cloud-init does not have any support for XenServer whatsoever. It is lacking any interfacing with XenStore. The first step would thus be to implement DataSource for cloud-init that supports reading network parameters from XenStore. This is somewhere on my long TODO list but will not come any time soon. If any good Python programmer is up for a challenge, I can offer some mentorship.
Good luck!