Refering the kafka documentation, a Kafka message producer configuration have request.timeout.ms
and timeout.ms
properties. Reading the description of these two properties, I am not able clearly distinguish between them.
Can anyone explain the difference using small example.
To handle the timeout exceptions, the general practice is: Rule out broker side issues. make sure that the topic partitions are fully replicated, and the brokers are not overloaded. Fix host name resolution or network connectivity issues if there are any.
linger.ms refers to the time to wait before sending messages out to Kafka. It defaults to 0, which the system interprets as 'send messages as soon as they are ready to be sent'. batch. size refers to the maximum amount of data to be collected before sending the batch.
max.block.ms is used for producer to block buffer time, serialization time etc. For details look at this one. https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient. Follow this answer to receive notifications.
request.timeout.ms
is the timeout configured on the client side. It says that the client is going to wait this much time for the server to respond to a request.
timeout.ms
is the timeout configured on the leader in the Kafka cluster. This is the timeout on the server side. For example if you have set the acks setting to all, the server will not respond until all of its followers have sent a response back to the leader. The leader will wait timeout.ms
amount of time for all the followers to respond.
So client sends a request to the server (leader). Based on the acks setting, the server will either wait or respond back to the client. timeout.ms
is the amount of time the leader waits for its followers whereas request.timeout.ms
is the amount of time the client waits for the server(leader).
They are both used for the common underlying network client, meaning the max amount of time for awaiting a response from the request, although timeout.ms is marked as 'Deprecated'. Actually, timeout.ms is only used on the producer side, and request.timeout.ms can be defined for both client(including producer and consumer) and server(for replicating threads).
Kafka recommends that user specify request.timeout.ms instead of timeout.ms
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With