azureiisazure-application-gatewaybad-gatewayhttp-status-code-502

Stopping Default IIS website causes Azure Application Gateway '502 Bad Gateway' Error for ALL websites in IIS


I'm having an issue with hosting multiple .NET websites on Windows Server/IIS and Azure Application Gateway.

We host multiple sites on a single Azure Windows VM running IIS, sitting behind Azure Application Gateway WAFv2. The VM is connected to App Gateway using a backend pool configured to point to the private IP of the VM, with the VNets peering configured between the App Gateway and VM VNets.

When I stop the default website in IIS, ALL websites then return a '502 Bad Gateway' error from Azure Application Gateway, and the backend health status changes to 'Unhealthy' for the backend pool where the VM resides.

Can anyone tell me why stopping the Default site would cause Application Gateway to error for all sites?

EDIT: Screenshot of IIS bindings as requested

EDIT 2: Apparently I can't answer my own question, however after working through this with our CSP I have the answer. By default the App Gateway Backend Health check looks at the default IIS site. If you stop that then the Backend Health Check fails and goes Unhealthy. At this point APP Gateway will no longer even ATTEMPT to route any requests, regardless of URL to that backend pool.


Solution

  • Found the answer to this after many months of messing about!

    With Azure Application Gateway, the default health probes for each backend pool ping and look for a response on the configured IP address or FQDN in the backend pool itself.

    In my case this is set to the local IP address of the Virtual machine (when I configured this 18-24 months ago I recall our Azure CSP telling me there was a bug with using the FQDN in the backend pool configuration).

    This means, that when the Health Probe is attempting to communicate with the VM, the Default Website in IIS is the only thing configured to respond to any requests on this IP address.

    If you stop the Default Site, the Health Probe gets no response to it's requests a the Backend Pool status goes to Unhealthy as you would expect.

    The really interesting thing here is that as soon as the Backend Pool Health Probe status goes Unhealthy, Azure Application Gateway ceases to even attempt to route any traffic to the affected backend pool. Instead it immediately reports the 502 Bad gateway error, and will continue to do so until the Health Probe status is corrected and goes back to healthy!