wso2wso2-api-managerapi-gatewaythrottlingrate-limiting

WSO2 API Manager 4.0.0 - Maximum Backend Throughput not working properly in a clustered environment

I am using WSO2 API Manager 4.0.0 in a clustered environment with:

I have set up a Maximum Backend Throughput limit for an API, but it seems that the throttling only kicks in when each Gateway node exceeds the limit, resulting in the backend receiving 10 times more traffic than expected.

According to the WSO2 API Manager documentation:

"The counters maintained when evaluating the maximum backend throughput are shared across all nodes of the Gateway cluster" (Source: WSO2 APIM 4.0.0 Documentation)

The Subscription Level Rate Limiting works correctly and is distributed among all Gateway nodes, indicating that the cluster is set up correctly and the rate limits are shared between the nodes.

However, the Maximum Backend Throughput limit does not seem to be applied correctly across the cluster. The throttling only happens when each individual Gateway node exceeds the limit, resulting in the backend receiving much more traffic than expected.

What could be causing this issue, and how can I ensure that the Maximum Backend Throughput limit is properly enforced across all Gateway nodes in the cluster?

Any help or guidance would be greatly appreciated. Thank you!

Solution

  • Both Subscription Level rate-limiting and Maximum Backend throughput works differently in the WSO2 APIM. Subscription level rate-limiting works by communicating to the Traffic Manager component and receiving the decision asynchronously to throttle the requests.

    The backend throughput limit works by sharing a counter between the Gateway components. However, in order for that to work, it is required to enable the Hazelcast clustering. Since, it has some limitations, it is good to split the TPS by the number of Gateway nodes that you have deployed in your environment and configuring the Backend throughput limit (if you are backend throughput limit is 100 TPS and if you are having 10 gateways deployed, then configure the backend throughput limit for the API as 10 TPS from the Publisher portal and deploy). With this, you should be able to achieve the backend throughput limit as expected.