Per Azure Front Door documentation, one of the ways to route traffic to the origins is based on latency. Given N origins and latency sensitivity of M, the calls will be sent to the subset of origins that have at most latency of M, and the other origins (that had latency > M) will not get any traffic.
The documentation explicitly mentions, under lowest latencies based traffic routing, setting M = 0, results in always forwarding the call to the origins with the lowest latency.
But what happens if the sensitivity is set to a positive value which is lower than all active origins' latency? This can happen due to network issues, or origins that had lower latency than M all failing. And suppose there are non failing origins, but they have latency higher than M. In such cases of failure, will the forwarding behaviour be the same as when M = 0? Or no calls will be forwarded to any of the origins till the failing origins recover.
The latency sensitivity setting works differently than what you have described. Latency sensitivity is not the threshold for routing traffic to an origin. We determine the threshold by adding the latency sensitivity value to the value of the fastest origin's latency.
The health probes periodically measure the latency of each origin. Traffic can be routed to an origin so long as its latency is under the threshold of "fastest origin's latency + latency sensitivity". (Assuming the other routing criteria are met, e.g. "healthy", "top priority") If the fasted origin has a latency of 50ms and the latency sensitivity is set to 20ms, then traffic can be routed to the origins with a latency of 70ms or less.
Even when the latency sensitivity is set to a lower value than the fastest origin traffic will still be routed to at least that fastest origin, since its latency is always under the threshold of "fastest origin latency + latency sensitivity".
The Microsoft Learn article traffic routing methods to origin has a good discussion of how Azure Front Door routing works and some useful links to drill deeper into how Azure Front Door determines which origins to route traffic to.