Run a single haproxy container in the swarm with a static IP on my LAN that will be recreated if a node fails. In essence, it would be akin to a VIP pointing to haproxy but without requiring an external load balancer outside of the swarm.
Example: Creating the macvlan networks on two nodes, running the container on node1, stopping the container, then running the container on node2. The haproxy container is created with the same static IP and is accessible on the LAN.
Example: Setting the macvlan network scope to swarm, then deploying the stack. Container is not visible to the network.
docker-compose.yml:
version: "3.2"
networks:
vlan0:
external: true
services:
haproxy:
image: haproxy:2.3.2-alpine
container_name: haproxy
volumes:
- ./data:/usr/local/etc/haproxy:ro
environment:
- TZ=America/Los_Angeles
restart: always
networks:
vlan0:
ipv4_address: 192.168.0.201
localnet.sh (script to stop stack / remove network / recreate network as local / run container local):
#!/bin/bash
docker service rm haproxy_haproxy
docker-compose down
docker network rm vlan0
docker network create -o parent=eth0 --subnet 192.168.0.0/24 --gateway 192.168.0.1 --driver macvlan --scope local vlan0
docker-compose up
swarmnet.sh (script to remove container and network / recreate network as swarm / run as swarm stack):
#!/bin/bash
docker service rm haproxy_haproxy
docker-compose down
docker network rm vlan0
docker network create -o parent=eth0 --subnet 192.168.0.0/24 --gateway 192.168.0.1 --driver macvlan --scope swarm vlan0
docker stack deploy -c docker-compose.yml haproxy
Any help is appreciated.
Edit: I have been able to get macvlan addresses working, but docker swarm does not obey the ipv4_address field to static the container. I understand the reason for this (replicas, etc. not going on the same IP) but in this scenario, it will not happen due to it being a single container. I've found this issue discussed here: https://forums.docker.com/t/docker-swarm-1-13-static-ips-for-containers/28060/
I've figured out how to bind a single static ip address (with a major caveat below).
To do this, create a config only network on each host with a single number ip range (32 bit mask). I've created a script here to auto-generate the commands.
#!/bin/bash
echo
echo VIP swarm command generator
echo
defint=$(ip route get 8.8.8.8 | head -n1 | awk '{print $5}')
hostip=$(ip addr show dev $defint | grep "inet" | awk 'NR==1{print $2}' | cut -d'/' -f 1)
hostsub=$(ip route | grep "src $hostip" | awk '{print $1}')
hostgate=$(ip route show | grep default | awk '{print $3}')
read -p "Subnet (default "$hostsub"): " sub
sub=${sub:=$hostsub}
read -p 'Gateway (default '$hostgate'): ' gate
gate=${gate:=$hostgate}
read -p 'Vip address: ' ip
last=`echo $ip | cut -d . -f 4`
begin=`echo $ip | cut -d"." -f1-3`
read -p 'Ethernet interface (default '$defint'): ' eth
eth=${eth:=$defint}
read -p 'Docker Network Name: (default vip-'$last'): ' name
name=${name:="vip-"$last}
echo -e "Build the network on each node, \033[0;33mmake sure the physical parent interface (parent=) is set properly on each node if different)\033[0m:"
echo -e " \033[44mdocker network create --config-only --subnet $sub --gateway $gate --ip-range $ip/32 -o parent=$eth $name\033[0m"
echo
echo "Then create the swarm network from any manager node:"
echo -e " \033[44mdocker network create -d macvlan --scope swarm --config-from $name swarm-$name\033[0m"
Example output:
VIP swarm command generator
Subnet (default 192.168.0.0/24):
Gateway (default 192.168.0.1):
Vip address: 192.168.0.201
Ethernet interface (default eth0):
Docker Network Name: (default vip-201):
Build the network on each node, make sure the physical parent interface (parent=) is set properly on each node if different):
docker network create --config-only --subnet 192.168.0.0/24 --gateway 192.168.0.1 --ip-range 192.168.0.201/32 -o parent=eth0 vip-201
Then create the swarm network from any manager node:
docker network create -d macvlan --scope swarm --config-from vip-201 swarm-vip-201
So, on each node, I build the network config
docker network create --config-only --subnet 192.168.0.0/24 --gateway 192.168.0.1 --ip-range 192.168.0.201/32 -o parent=eth0 vip-201
Then I build the swarm interface
docker network create -d macvlan --scope swarm --config-from vip-201 swarm-vip-201
And in the applicable docker-compose.yaml file, add the appropriate lines:
networks:
swarm-vip-201:
external: true
services:
haproxy:
...
networks:
swarm-vip-201:
My container will now always be available at that single static IP address. If the node fails, it will restart on another node with the same IP, in essence creating a high-availability VIP for the swarm.
Docker swarm will throw errors trying to bind multiple macvlans to the same parent interface, like below:
"network NETNAME is already using parent interface eth0"
Described here:
https://github.com/moby/libnetwork/issues/2384
https://github.com/moby/libnetwork/issues/1743
It is promising that it is actually possible to create a static macvlan IP in the swarm, but until the bug is fixed with binding multiple networks to a single parent interface, it is very limited.
If anyone knows how to reliably fix the problem above, or has a better method, please post.