multicastwindows-server-2019wfpwindows-networking

Why does Windows Filtering Platform apply ALE reauthorization for EVERY single incoming multicast UDP packet from same source?


enter image description hereI am using iperf network testing tool and observe the following: if, on box A (Windows Server 2019), I have a single sender that posts to some multicast address/port, and 5 multicast receivers, joining same multicast group, on box B (also Windows Server 2019), there will be packet loss at certain packet rate, which does NOT happen in case of single receiver, so it is NOT sender or wire level loss

Exploring with xperf (stackwalk with DPC details) shows that there is some relevant activity in ntoskrnl.exy, but in netio.sys, it points to tcpip.sys!WfpAleReauthorizeConnection , which is called from tcpip.sys!TlShimOptionalReauthorizeConnection. The part "optional" feels like it's not something that HAS TO happen. Is there a way to control it and make it to not take that option?

Looking at the WFP performance counters, I see that counter "Reason:NewInboundMCastBCastPacket" (WFP Reauthorization counter group) value is exactly equal to the number of UDP datagram processed (input stream packet rate multiplied by 5, because 5 receivers). That tells me Reauthorization is indeed applied to every single packet. There has to be a way to control this. The issue is not limited to testing tool - we have an actual application, which is also "5 copies of multicast receivers" that is running into same exact issue.

Windows Defender Firewall is off, so, it is not something that it is doing. Switching network profile from "public" to "private". Adding a filter to explicitly permit all UDP traffic at FWPM_LAYER_ALE_AUTH_RECV_ACCEPT_V4 layer did not change anything - I am guessing I need to somehow adjust filtering engine to not even try to apply filtering, but WFP documentation is extremely vague , and there is NO mentioning of NewInboundMCastBCastPacket as a possible cause of WFP ALE reauthorization

Tried the following: Completely uninstalled Symantec Endpoint Protection Tried to add FWP_ACTION_PERMIT at FWPM_LAYER_ALE_AUTH_RECV_ACCEPT_V4 layer to allow ALL inbound UDP traffic (but I think I need a way to force FWP engine to not even apply the checking) Switched network profile from "Public" to "Private"


Solution

  • Turns out there is a way to exempt multicast traffic from inspection by UDP port:

    You can create a REG_MULTI_SZ type registry key named UdpExemptPortRange to configure the port ranges that are exempted from the base filtering engine. To do this, create the registry key in the following path:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\services\Tcpip\Parameters

    For example, you can configure the following value in the registry key:

    • 3400
    • 1000-1030
    • 2000-2050

    The smallest port number to use is 1 and the highest is 65535.

    Notes

    • Each exempted port range should be in a new line.
    • You must restart the computer after you changed the registry key.