Menu
In kafka producer settings, what is the expected behavior if you have linger.ms set to 0 and a non zero batch.size? How long will the producer wait for batch.size to be reached before sending the messages? Will it wait forever till the size of messages is less than specified batch size OR since linger.ms is zero, it will not do any batching and just send every request?
576
The default is 16KB . Increasing a batch size to 32KB or 64KB can help increase the compression, throughput, and efficiency of requests. Any message that is bigger than the batch size will not be batched.
It controls how many bytes of data to collect before sending messages to the Kafka broker. Set this as high as possible, without exceeding available memory. The default value is 16384. When you use Producer.
linger.ms refers to the time to wait before sending messages out to Kafka. It defaults to 0, which the system interprets as 'send messages as soon as they are ready to be sent'. batch. size refers to the maximum amount of data to be collected before sending the batch.
No, this does not mean that the producer will wait for the batch to become full. The producer will send half-full batches and even batches with just a single message in them.
Therefore, setting the batch size too large will not cause delays in sending messages; it will just use more memory for the batches. Setting the batch size too small will add some overhead because the producer will need to send messages more frequently.
By default (linger.ms=0), the producer will send messages as soon as there is a sender thread available to send them, even if there’s just one message in the batch.
Hope it helps.
Thanks.
73
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
