Java Throttling

Question

How do I use a combination of ScheduledThreadPoolExecutor, ScheduledFuture and ExecutorCompletionService to throttle Callable commands that accept a variable parameter? Upon receiving a response from a Callable command, I need to create a new Callable command based on the output of the aforementioned Callable command. I also need to adhere to a threshold of 100 calls per second.

Answer 1

You should implement the Leaky Bucket algorithm. Before making a call, block until you have a token. You can implement this algorithm in a few dozen lines of Java.

Answer 2

I would suggest using a broker, for example RabbitMQ. You could setup the maximum number of consumers to 100 and have a single Producer instance which publishes at a rate of 100 messages per second.

Here you can find an explanation of three methods for implementing a throttle mechanism in a distributed system. The one you will be interested in playing with is the distributed which uses RabbitMQ. This one is designed to limit the number of concurrent messages at any given time, let's say at most 100 at any given time. You would need to modify it so the publisher publishes no more than 100 messages per second. At the bottom you can find an url to the git repository with the source code but anyway, I'm pasting it as well here.

Edit from Comments:

First uses java.util.Semaphore which is configured with the number of permits it will handle. Each thread will try to acquire a permit and will be blocked if there are no permits left until one is freed. Second one uses a fixed size ThreadPoolExecutor. The executor will have at most the specified number of working threads at any given time. Third one uses RabbitMQ. The maximum number of concurrent consumers would be the maximum number of working threads. The git repo has a more detailed explanation in English. Hope this helps