dockerdocker-swarmmacvlan

Docker Macvlan network inside container is not reaching to its own host

I have setup Macvlan network between 2 docker host as follows:

Host Setup: HOST_1 ens192: 172.18.0.21

Create macvlan bridge interface

docker network  create  -d macvlan \
--subnet=172.18.0.0/22 \
--gateway=172.18.0.1 \
--ip-range=172.18.1.0/28 \
-o macvlan_mode=bridge \
-o parent=ens192 macvlan

Create macvlan interface HOST_1

ip link add ens192.br link ens192 type macvlan mode bridge
ip addr add 172.18.1.0/28 dev ens192.br
ip link set dev ens192.br up

Host Setup: HOST_2 ens192: 172.18.0.23

Create macvlan bridge interface

docker network  create  -d macvlan \
--subnet=172.18.0.0/22 \
--gateway=172.18.0.1 \
--ip-range=172.18.1.16/28 \
-o macvlan_mode=bridge \
-o parent=ens192 macvlan

Create macvlan interface in HOST_2

ip link add ens192.br link ens192 type macvlan mode bridge
ip addr add 172.18.1.16/28 dev ens192.br
ip link set dev ens192.br up

Container Setup

Create containers in both host

HOST_1# docker run --net=macvlan -it --name macvlan_1 --rm alpine /bin/sh
HOST_2# docker run --net=macvlan -it --name macvlan_1 --rm alpine /bin/sh

CONTAINER_1 in HOST_1

24: eth0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 02:42:ac:12:01:00 brd ff:ff:ff:ff:ff:ff
    inet 172.18.1.0/22 brd 172.18.3.255 scope global eth0
       valid_lft forever preferred_lft forever

CONTAINER_2 in HOST_2

21: eth0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 02:42:ac:12:01:10 brd ff:ff:ff:ff:ff:ff
    inet 172.18.1.16/22 brd 172.18.3.255 scope global eth0
       valid_lft forever preferred_lft forever

Route table in CONTAINER_1 and CONTAINER_2

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.18.0.1      0.0.0.0         UG    0      0        0 eth0
172.18.0.0      0.0.0.0         255.255.252.0   U     0      0        0 eth0

Scenario

HOST_1 (172.18.0.21) <-> HOST_2 (172.18.0.23) = OK (Vice-versa)

HOST_1 (172.18.0.21) -> CONTAINER_1 (172.18.1.0) and CONTAINER_2 (172.18.1.16) = OK

HOST_2 (172.18.0.23) -> CONTAINER_1 (172.18.1.0) and CONTAINER_2 (172.18.1.16) = OK

CONTAINER_1 (172.18.1.0) -> HOST_2 (172.18.0.23) = OK

CONTAINER_2 (172.18.1.16) -> HOST_1 (172.18.0.21) = OK

CONTAINER_1 (172.18.1.0) <-> CONTAINER_2 (172.18.1.16) = OK (Vice-versa)

CONTAINER_1 (172.18.1.0) -> HOST_1 (172.18.0.21) = FAIL

CONTAINER_2 (172.18.1.16) -> HOST_2 (172.18.0.23) = FAIL

Question

I am very close to my solution I wanted to achieve except this 1 single problem. How can I make this work for container to connect to its own host. If there is solution to this, I would like to know how to configure in ESXi virtualization perspective and also bare-metal if there is any difference

Solution

  • This is defined behavior for macvlan and is by design. See Docker Macvlan Documentation

    • When using macvlan, you cannot ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host’s eth0, it will not work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

    • A macvlan subinterface can be added to the Docker host, to allow traffic between the Docker host and containers. The IP address needs to be set on this subinterface and removed from the parent address.