azurenetwork-programmingazure-virtual-machinenicazure-public-ip

New NIC with public IP has no internet connection, while existing NIC works fine


I'm experiencing an issue with a newly added network interface (NIC) on my Azure VM. While the existing NIC works perfectly, the new one fails to connect to the internet. Here are the details:

Current Setup:

Problem:

  1. Added a new NIC with a new public IP to the VM

    (eth1 is a newly added nic.)
    azureuser@instanceXX:~$ ip addr show
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
        link/ether 60:45:bd:48:e7:f1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.0.5/24 brd 10.0.0.255 scope global eth0
           valid_lft forever preferred_lft forever
     # ~~~~skipping the middle~~~~ 
        inet6 fe80::6245:bdff:fe48:e7f1/64 scope link
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
        link/ether 7c:1e:52:2b:a9:ff brd ff:ff:ff:ff:ff:ff
        inet 10.0.0.104/24 metric 200 brd 10.0.0.255 scope global eth1
           valid_lft forever preferred_lft forever
        inet6 fe80::7e1e:52ff:fe2b:a9ff/64 scope link
           valid_lft forever preferred_lft forever
    4: enP12745s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP group default qlen 1000
        link/ether 60:45:bd:48:e7:f1 brd ff:ff:ff:ff:ff:ff
        altname enP12745p0s2
        inet6 fe80::6245:bdff:fe48:e7f1/64 scope link
           valid_lft forever preferred_lft forever
    

    Looking at # route -n I get this output:

    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    0.0.0.0         10.0.0.1        0.0.0.0         UG    100    0        0 eth0
    10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
    10.0.0.0        0.0.0.0         255.255.255.0   U     200    0        0 eth1
    10.0.0.1        0.0.0.0         255.255.255.255 UH    100    0        0 eth0
    168.63.129.16   10.0.0.1        255.255.255.255 UGH   100    0        0 eth0
    168.63.129.16   0.0.0.0         255.255.255.255 UH    200    0        0 eth1
    169.254.169.254 10.0.0.1        255.255.255.255 UGH   100    0        0 eth0
    
  2. Both Primary and Secondary type IPs on the new NIC fail to connect:

    azureuser@instanceXX:~$ curl --interface 10.0.0.104 http://example.com
    curl: (28) Failed to connect to example.com port 80 after 134224 ms: Connection timed out
    
    
  3. Existing NIC and its IPs still work correctly

  4. Both NICs share the same Network Security Group (NSG), which hasn't been modified since VM creation

Expected Behavior: Two NICs, each with 225 public IPs (Basic SKU, dynamic), all functioning normally.

Additional Notes:

Has anyone encountered a similar issue or can suggest a solution to get the new NIC working with internet connectivity? Any insights would be greatly appreciated...!!!

Attempted Solutions:

  1. Used Azure's Connection troubleshoot and Support + troubleshooting tools (no resolution)

  2. Tried creating IPs with different SKUs, but VM failed to boot due to SKU mismatch

  3. Created a new VM and replicated the setup, but encountered the same issue


Solution

  • To address the issue of a newly added network interface on your Azure VM not being able to connect to the internet while the existing NIC works fine, follow these steps-

    Create a Virtual Network and Subnet

    az network vnet create \
      --resource-group arkorg \
      --name myVNet \
      --address-prefix 10.0.0.0/16 \
      --subnet-name mySubnet \
      --subnet-prefix 10.0.0.0/24
    

    enter image description here

    Create a Network Security Group (NSG)

    az network nsg create \
      --resource-group arkorg \
      --name myNSG
    

    enter image description here

    Add Inbound Rules to the NSG for HTTP and SSH

    az network nsg rule create \
      --resource-group arkorg \
      --nsg-name myNSG \
      --name AllowInternetInBound \
      --priority 1000 \
      --direction Inbound \
      --access Allow \
      --protocol Tcp \
      --destination-port-range 80 \
      --source-address-prefix Internet \
      --destination-address-prefix '*'
    
    az network nsg rule create \
      --resource-group arkorg \
      --nsg-name myNSG \
      --name AllowSSH \
      --priority 1100 \
      --direction Inbound \
      --access Allow \
      --protocol Tcp \
      --destination-port-range 22 \
      --source-address-prefix Internet \
      --destination-address-prefix '*'
    

    enter image description here

    enter image description here

    Create Public IP Addresses for the NICs

    az network public-ip create \
      --resource-group arkorg \
      --name myExistingPublicIP \
      --sku Basic \
      --allocation-method Dynamic
    
    az network public-ip create \
      --resource-group arkorg \
      --name myNewPublicIP \
      --sku Basic \
      --allocation-method Dynamic
    

    enter image description here

    enter image description here

    Create Network Interfaces and Associate Them with Public IPs

    az network nic create \
      --resource-group arkorg \
      --name myExistingNIC \
      --vnet-name myVNet \
      --subnet mySubnet \
      --network-security-group myNSG \
      --public-ip-address myExistingPublicIP
    
    az network nic create \
      --resource-group arkorg \
      --name myNewNIC \
      --vnet-name myVNet \
      --subnet mySubnet \
      --network-security-group myNSG \
      --public-ip-address myNewPublicIP
    

    enter image description here

    enter image description here

    Create a VM with your existing NIC

    az vm create \
      --resource-group arkorg \
      --name myVM \
      --nics myExistingNIC \
      --image Ubuntu2204 \
      --admin-username azureuser \
      --generate-ssh-keys
    

    enter image description here

    Now comes your main problem that is update the VM with new IP and it should be able to connect to the net.

    So first deallocate the old one

    az vm deallocate \
      --resource-group arkorg \
      --name myVM
    

    enter image description here

    Followed by adding of your new NIC and restarting the VM

    az vm nic add \
      --resource-group arkorg \
      --vm-name myVM \
      --nics myNewNIC
    
    az vm start \
      --resource-group arkorg \
      --name myVM
    

    enter image description here

    enter image description here

    If done till here, then you're sorted. Now you just have to SSH into the VM using the public IP of the existing NIC

    ssh azureuser@<existing-public-ip>
    

    Run the following commands to set up source-based routing:

    sudo su
    echo "200 eth0" >> /etc/iproute2/rt_tables
    echo "201 eth1" >> /etc/iproute2/rt_tables
    ip rule add from 10.0.0.4/32 table eth0
    ip rule add from 10.0.0.5/32 table eth1
    ip route add 10.0.0.0/24 dev eth0 src 10.0.0.4 table eth0
    ip route add default via 10.0.0.1 dev eth0 table eth0
    ip route add 10.0.0.0/24 dev eth1 src 10.0.0.5 table eth1
    ip route add default via 10.0.0.1 dev eth1 table eth1
    

    enter image description here

    Verify the routing rules

    ip rule show
    ip route show table eth0
    ip route show table eth1
    

    enter image description here

    Test Connectivity

    curl --interface 10.0.0.4 http://example.com
    curl --interface 10.0.0.5 http://example.com
    

    enter image description here