What's a good way to enforce a single rate limit on multiple machines?

Question

I have a web service with a load balancer that maps requests to multiple machines. Each of these requests end up sending a http call to an external API, and for that reason I would like to rate limit the number of requests I send to the external API.

My current design:

• Service has a queue in memory that stores all received requests
• I rate limit how often we can grab a request from the queue and process it.

This doesn't work when I'm using multiple machines, because each machine has its own queue and rate limiter. For example: when I set my rate limiter to 10,000 requests/day, and I use 10 machines, I will end up processing 100,000 requests/day at full load because each machine processes 10,000 requests/day. I would like to rate limit so that only 10,000 requests get processed/day, while still load balancing those 10,000 requests.

I'm using Java and MYSQL.

Answer 1

1. use memcached or redis keep api request counter per client. check every request if out rate limit.
2. if you think checking at every request is too expensive,you can try storm to process request log, and async calculate request counter.