How to restrict direct access from internet to Azure Public LoadBalancer backend pool VM with NSG

As the question at title, I'm setup the following architecture on Azure Cloud and having trouble at restricting direct access from the internet to VMs.

Here are architecture requirements:

• Both VMs must have public ips (for SysAdmin to access via SSH)
• Direct traffics from Internet to WebService on VMs (via port 80) must be denied
• The web traffics from Internet must go thru Public LB to VMs

Suppose that both VMs are in WebASG (Application Security Group), in the NSG setting that applied to VM's Subnet, I've add some rules (which have higher priority than 3 Azure NSG default rules):

1. Scenario A (adding 1 custom rule):

Port: 80 - Protocol: Tcp - Source: Internet - Destination: WebASG - Action: Allow

With this NSG setting, I could access WebService from LoadBalancer IP (satisfy #3 requirement), but WebService on port 80 of both VMs will be exposed to Internet (that violates #2 requirement)

2. Scenario B (adding 2 custom rules):

Port: 80 - Protocol: Tcp - Source: AzureLoadBalancer - Destination: WebASG - Action: Allow

Port: 80 - Protocol: Tcp - Source: Internet - Destination: WebASG - Action: Deny

With this NSG setting, #2 requirement is satisfied, but I could not access WebService when visit LoadBalancer IP (violates #3 requirement)

Please note that: using AGW (Azure Application Gateway, I could make all the requirements happened by these NSG configuration:

RuleName: AllowSSH Port: 22 - Protocol: Tcp - Source: sys-admin-ip-address - Destination: WebASG - Action: Allow

RuleName: DenyInternet2Web Port: Any - Protocol: Any - Source: Internet - Destination: WebASG - Action: Deny

RuleName: AllowProbe2Web Port: 80 - Protocol: Tcp - Source: VirtualNetwork - Destination: WebASG - Action: Allow

RuleName: AllowProbe2Web Port: 80 - Protocol: Tcp - Source: VirtualNetwork - Destination: WebASG - Action: Allow

I dont want using AGW because it would cost more money than Azure LoadBalancer (actually the Basic LoadBalancer is free). So, how could I change NSG to satisfy all requirements when using LoadBalancer?

Thank in advance for any help!

I don't think there are NSG rules that will satisfy all requirements because of the #1 and #2 requirements are contradictory.

If the VMs must have public IP addresses, it actually has a chance to expose to the Internet. Any clients could access the VMs via the public IP. It works the same if you want to access the VMs through the load balancer frontend IP. Read the https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-overview#load-balancer-concepts

Load Balancer doesn't terminate or originate flows, interact with the payload of the flow, or provide any application layer gateway function. Protocol handshakes always occur directly between the client and the back-end pool instance. A response to an inbound flow is always a response from a virtual machine. When the flow arrives on the virtual machine, the original source IP address is also preserved.

In this case, you could remove the backend-instance IP address, just use the load balancer frontend for the web traffic and SSH connection. If so, You could configure port forwarding in Azure Load Balancer for the SSH connections to individual instances and a load balancer rule for the web traffic following this quickstart, which works with standard LB. You can only allow port 80 and 22 from your clients' IP addresses. The NSG will look like this,

Port: 80,22 - Protocol: Tcp - Source: client's IP list - Destination: WebASG - Action: Allow