How to implement distributed rate limiter?

Question

Let's say, I have P processes running some business logic on N physical machines. These processes call some web service S, say. I want to ensure that not more than X calls are made to the service S per second by all the P processes combined.

How can such a solution be implemented?

Google Guava's Rate Limiter works well for processes running on single box, but not in distributed setup.

Are there any standard, ready to use, solutions available for JAVA? [may be based on zookeeper]

Thanks!

Answer 1

Bucket4j is java implementation of "token-bucket" rate limiting algorithm. It works both locally and distributed(on top of JCache). For distributed use case you are free to choose any JCache implementation like Hazelcast or Apache Ignite. See this example of using Bucket4j in cluster.

Answer 2

I have been working on an opensource solution for these kind of problems.

Limitd is a "server" for limits. The limits are implemented using the Token Bucket Algorithm.

Basically you define limits in the service configuration like this:

buckets:
  "request to service a":
     per_minute: 10
  "request to service b":
     per_minute: 5

The service is run as a daemon listening on a TCP/IP port.

Then your application does something along these lines:

var limitd = new Limitd('limitd://my-limitd-address');

limitd.take('request to service a', 'app1' 1, function (err, result) {
  if (result.conformant) {
    console.log('everything is okay - this should be allowed');
  } else {
    console.error('too many calls to this thing');
  }
});

We are currently using this for rate-limiting and debouncing some application events.

The server is on:

https://github.com/auth0/limitd

We are planning to work on several SDKs but for now we only have node.js and partially implemented go:

https://github.com/limitd