networkingopenstackbridge

Trouble creating a bridge device for OpenStack Rocky Linux 9.2 networking


Problems encountered when create the network bridge (OpenStack)

I'm currently building openstack according to the official Openstack-ansible documentation, which can be found here: https://docs.openstack.org/project-deploy-guide/openstack-ansible/zed/targethosts.html So, I'm building a bridge in Rocky Linux 9.2. Due to the use of virtual machines for setup tests, the external network to which my virtual machine is connected has already entered the vlan, so now I just need to create a bridge device (or several) inside each node host and use bridge-slave to connect this bridge device with a network interface (ens160 in my case) that can access the Internet. So, I executed the following command:

nmcli c add type bridge autoconnect yes ifname br-mgmt con-name br-mgmt

nmcli c

There are the following echoes:

NAME     UUID                                  TYPE      DEVICE  
br-mgmt  2b4c8058-f6ab-471f-afdb-e110cfdcc550  bridge    br-mgmt 
ens160   ce68d146-f671-3eec-b26f-de540af181a1  ethernet  ens160  
lo       5d8a90ae-138c-4544-a415-dd0e3c23050f  loopback  lo

This indicates that the creation was successful. Then, run the following command to create the bridge slave: nmcli con add type bridge-slave ifname ens160 master br-mgmt

If continue executing nmcli c, we get the following echo:

NAME                 UUID                                  TYPE      DEVICE
br-mgmt              2b4c8058-f6ab-471f-afdb-e110cfdcc550  bridge    br-mgmt
ens160               ce68d146-f671-3eec-b26f-de540af181a1  ethernet  ens160
lo                   5d8a90ae-138c-4544-a415-dd0e3c23050f  loopback  lo
bridge-slave-ens160  5f3fc816-406c-4cc0-8d6d-1d68841a6d55  ethernet  --

When you execute the command nmcli c up br-mgmt, you get the following echo:

Connection successfully activated (master waiting for slaves) (D-Bus active path:  /org/freedesktop/NetworkManager/ActiveConnection/22)

Then, we can check the network status using the ip addr command:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:62:bb:02 brd ff:ff:ff:ff:ff:ff
    altname enp3s0
    inet 192.168.75.141/24 brd 192.168.75.255 scope global dynamic noprefixroute ens160
       valid_lft 1006sec preferred_lft 1006sec
    inet6 fe80::20c:29ff:fe62:bb02/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
16: br-mgmt: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether c2:51:e9:6f:47:f4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.48.1/24 brd 192.168.48.255 scope global noprefixroute br-mgmt
       valid_lft forever preferred_lft forever

This indicates that the bridge device br-mgmt is still in the down state and cannot be started. Even if you use the nmcli c up command to start.

Check the file /etc/NetworkManager/system-connections/br-mgmt.nmconnection:

connection.id:                          br-mgmt
connection.id:                          br-mgmt
connection.uuid:                        2b4c8058-f6ab-471f-afdb-e110cfdcc550
connection.stable-id:                   --
connection.type:                        bridge
connection.interface-name:              br-mgmt
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1685948786
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
connection.dns-over-tls:                -1 (default)
connection.mptcp-flags:                 0x0 (default)
connection.wait-device-timeout:         -1
connection.wait-activation-delay:       -1
802-3-ethernet.port:                    --
802-3-ethernet.speed:                   0
802-3-ethernet.duplex:                  --
802-3-ethernet.auto-negotiate:          no
802-3-ethernet.mac-address:             --
802-3-ethernet.cloned-mac-address:      --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist:   --
802-3-ethernet.mtu:                     auto
802-3-ethernet.s390-subchannels:        --
802-3-ethernet.s390-nettype:            --
802-3-ethernet.s390-options:            --
802-3-ethernet.wake-on-lan:             default
802-3-ethernet.wake-on-lan-password:    --
802-3-ethernet.accept-all-mac-addresses:-1 (default)
ipv4.method:                            manual
ipv4.dns:                               223.5.5.5
ipv4.dns-search:                        --
ipv4.dns-options:                       --
ipv4.dns-priority:                      0
ipv4.addresses:                         192.168.48.1/24
ipv4.gateway:                           192.168.48.1
ipv4.routes:                            --
ipv4.route-metric:                      -1
ipv4.route-table:                       0 (unspec)
ipv4.routing-rules:                     --
ipv4.replace-local-rule:                -1 (default)
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-iaid:                         --
ipv4.dhcp-timeout:                      0 (default)
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.dhcp-fqdn:                         --
ipv4.dhcp-hostname-flags:               0x0 (none)
ipv4.never-default:                     no
ipv4.may-fail:                          yes
ipv4.required-timeout:                  -1 (default)
ipv4.dad-timeout:                       -1 (default)
ipv4.dhcp-vendor-class-identifier:      --
ipv4.link-local:                        0 (default)
ipv4.dhcp-reject-servers:               --
ipv4.auto-route-ext-gw:                 -1 (default)
ipv6.method:                            auto
ipv6.dns:                               --
ipv6.dns-search:                        --
ipv6.dns-options:                       --
ipv6.dns-priority:                      0
ipv6.addresses:                         --
ipv6.gateway:                           --
ipv6.routes:                            --
ipv6.route-metric:                      -1
ipv6.route-table:                       0 (unspec)
ipv6.routing-rules:                     --
ipv6.replace-local-rule:                -1 (default)
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.required-timeout:                  -1 (default)
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.addr-gen-mode:                     default
ipv6.ra-timeout:                        0 (default)
ipv6.mtu:                               auto
ipv6.dhcp-duid:                         --
ipv6.dhcp-iaid:                         --
ipv6.dhcp-timeout:                      0 (default)
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --
ipv6.dhcp-hostname-flags:               0x0 (none)
ipv6.auto-route-ext-gw:                 -1 (default)
ipv6.token:                             --
bridge.mac-address:                     --
bridge.stp:                             yes
bridge.priority:                        32768
bridge.forward-delay:                   15
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           --
proxy.method:                           none
proxy.browser-only:                     no
proxy.pac-url:                          --
proxy.pac-script:                       --
GENERAL.NAME:                           br-mgmt
GENERAL.UUID:                           2b4c8058-f6ab-471f-afdb-e110cfdcc550
GENERAL.DEVICES:                        br-mgmt
GENERAL.IP-IFACE:                       br-mgmt
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        no
GENERAL.DEFAULT6:                       no
GENERAL.SPEC-OBJECT:                    --
GENERAL.VPN:                            no
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/22
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/14
GENERAL.ZONE:                           --
GENERAL.MASTER-PATH:                    --
IP4.ADDRESS[1]:                         192.168.48.1/24
IP4.GATEWAY:                            192.168.48.1
IP4.ROUTE[1]:                           dst = 192.168.48.0/24, nh = 0.0.0.0, mt = 425
IP4.ROUTE[2]:                           dst = 0.0.0.0/0, nh = 192.168.48.1, mt = 425
IP4.DNS[1]:                             223.5.5.5
IP6.GATEWAY:                            --

Solution

  • Hey there :) I'm part of the OpenStack-Ansible project and help maintain Rocky Linux in the project.

    Firstly, if you have not yet tried setting up an 'All-In-One', I would strongly suggest you try that out on a virtual machine first. This will give you a great introduction to the project and how all the pieces fit together. OpenStack-Ansible (OSA) is a deployment model for OpenStack, which is complex in and of itself, so it can be good to get comfortable with the components before trying to perform a multi-node installation. Of note, it's also possible to have a multi-node 'All-In-One' (AIO) which can also be beneficial for learning. We use this in our CI testing to validating components which need to exist on multiple nodes. More info about this can be found in the openstack-ansible-ops repository, along with many more examples and information.

    For your question about bridges: As of the Zed release, there are two ways to do this. In Zed, the default Neutron plugin switched from LinuxBridge to OVN--Open Virtual Network, which can be used to delegate the configuration of the host networking to OpenStack-Ansible. In any case, OSA does not mandate any specific method to configure the interfaces, as long as they are configured and functional on the target hosts.

    In your specific instance, it appears that your bridge is down because another NetworkManager interface has already preempted the physical device:

    NAME                 UUID                                  TYPE      DEVICE
    br-mgmt              2b4c8058-f6ab-471f-afdb-e110cfdcc550  bridge    br-mgmt
    ens160               ce68d146-f671-3eec-b26f-de540af181a1  ethernet  ens160
    lo                   5d8a90ae-138c-4544-a415-dd0e3c23050f  loopback  lo
    bridge-slave-ens160  5f3fc816-406c-4cc0-8d6d-1d68841a6d55  ethernet  --
    

    Here we see the ens160 Connection has already taken the ens160 device, meaning the bridge-slave-ens160 Connection cannot enslave to br-mgmt.

    For an example of how I setup the bridges and bonds in my environment, you can checkout the Ansible Playbook and associated vars file, though this is a work in progress as I am migrating to OSA-Managed interfaces.

    If you have any more questions, please feel free to comment and/or reach out on IRC. OpenStack-Ansible is in the #openstack-ansible channel on ofct.net