What is the difference between replenishRate and burstCapacity?

Question

In the Redis implementation of the RequestRateLimiter, we must specify two properties redis-rate-limiter.replenishRate and redis-rate-limiter.burstCapacity as arguments for the RequestRateLimiter filter.

According to the documentation,

The redis-rate-limiter.replenishRate is how many requests per second do you want a user to be allowed to do, without any dropped requests. This is the rate that the token bucket is filled.

The redis-rate-limiter.burstCapacity is the maximum number of requests a user is allowed to do in a single second. This is the number of tokens the token bucket can hold. Setting this value to zero will block all requests.

From what I see, replenishRate is the rate at which the requests are being made, and the burstCapacity is the maximum requests that can be made (both under one second). However, I can't seem to understand the difference between the two in a practical scenario.

Answer 1

It's easier to grasp with different time units, e.g:

• replenish rate: 1000 requests per minute
• burst capacity: 500 requests per second

The former controls that you never get more than 1000 requests in a minute while the latter allows you to support temporary load peaks of up to 500 requests in the same second. You could have one 500 burst in second 0, another 500 burst in second 1 and you would've reached the rate limit (1000 requests within the same minute), so new requests in the following 58 seconds would be dropped.

In the context of Spring Cloud Gateway (SCG) the documentation is kind of ambiguous (the rate limiter needs to be allowed some time...):

A steady rate is accomplished by setting the same value in replenishRate and burstCapacity. Temporary bursts can be allowed by setting burstCapacity higher than replenishRate. In this case, the rate limiter needs to be allowed some time between bursts (according to replenishRate), as two consecutive bursts will result in dropped requests (HTTP 429 - Too Many Requests).

Extrapolating from the previous example I'd say that SCG works like this:

• replenish rate: 1000 requests per second
• burst capacity: 2000 requests per second

You are allowed to have a burst (peak) of 2000 requests in the same second (second 0). Since your replenish rate is 1000 rps, you've already passed two cycles' allowance so you couldn't send another message until second 3.