How do Kafka Quotas work?

Question

My question is: If I set my quota for 1 MB/sec, How much does a multi-broker cluster get? (For Example, take 2 brokers) Will the speed be 1MB/sec per each broker (or) 0.5 MB/sec for each broker?

The documentation says:

For example, if (user="test-user", client-id="test-client") has a produce quota of 10MB/sec, this is shared across all producer instances of user "test-user" with the client-id "test-client.

Answer 1

Yes Quotas are calculated per broker. So in your example, you will get 1MB/sec on each broker.

The Quotas Enforcement section in the Kafka documentation says:

By default, each unique client group receives a fixed quota as configured by the cluster. This quota is defined on a per-broker basis. Each client can utilize this quota per broker before it gets throttled. We decided that defining these quotas per broker is much better than having a fixed cluster wide bandwidth per client because that would require a mechanism to share client quota usage among all the brokers. This can be harder to get right than the quota implementation itself!