We are looking at 2 different libraries that work in the same way: Apache Guava RateLimiter & Resilience4J.
According to the documentation:
RateLimiter uses the token bucket algorithm and accumulates the tokens. If the token consumption frequency is low, the requests can directly get the tokens without waiting. RateLimiter accumulates tokens to allow it to cope with a sudden increase in traffic. When RateLimiter does not have sufficient number of tokens available, it uses the delayed processing method.
I observe that at such a moment the number of resolutions can reach x2 of the given one per second. Here is a simplified example of the Guava RateLimiter issue:
public static void main(String[] args) throws InterruptedException {
var result = new ArrayList<Integer>();
var rateLimiter = RateLimiter.create(5);
// Required to accumulate permissions
Thread.sleep(30000);
for (int i = 0; i < 30; i++) {
result.add(LocalTime.now().getSecond());
rateLimiter.acquire();
}
var collect = result.stream().collect(Collectors.groupingBy(e -> e, Collectors.counting()));
System.out.println(collect);
}
The result will be something like this:
{42=10, 43=5, 44=5, 45=5, 46=5}
.
X2 permissions at the first second instead of target 5.
So my main question is: how to disable such logic with accumulation? Or what are the options to avoid this? I can't cross the 5 rps limit.
P.S. Resilience4J works with the same logic, but creation should be:
public static RateLimiter buildRateLimiter() {
var config = RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(1))
.limitForPeriod(5)
.build();
return RateLimiterRegistry.of(config).rateLimiter("test", config);
}
We found a solution only using Resilience4J. Based on the fact that the accumulation of the bucket in the rate limiter works on the basis of periodicity - we just redefined the period setting.
Instead of 5 requests per 1 second, it became 1 request per 0.2s.
More precisely, it is even a smoothing of the effect of the accumulation of tokens instead of a solution, but it works pretty well.
public static RateLimiter buildRateLimiter(Duration period, int limit) {
var config = RateLimiterConfig.custom()
.limitRefreshPeriod(ofMillis(period.toMillis() / limit))
.limitForPeriod(1)
.build();
return RateLimiterRegistry.of(config).rateLimiter("test", config);
}