Why can't I increase session.timeout.ms?

Question

I want to increase session.timeout.ms to allow longer time for processing the messages received between poll() calls. However when I change session.timeout.ms to a higher value than 30000, it fails to create Consumer object and throws below error.

Could anyone tell why can't I increase session.timeout.ms value or if I am missing something?

0    [main] INFO  org.apache.kafka.clients.consumer.ConsumerConfig  - ConsumerConfig values:   request.timeout.ms = 40000 check.crcs = true retry.backoff.ms = 100 ssl.truststore.password = null ssl.keymanager.algorithm = SunX509 receive.buffer.bytes = 262144 ssl.cipher.suites = null ssl.key.password = null sasl.kerberos.ticket.renew.jitter = 0.05 ssl.provider = null sasl.kerberos.service.name = null session.timeout.ms = 40000 sasl.kerberos.ticket.renew.window.factor = 0.8 bootstrap.servers = [server-name:9092] client.id =  fetch.max.wait.ms = 500 fetch.min.bytes = 50000 key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer sasl.kerberos.kinit.cmd = /usr/bin/kinit auto.offset.reset = latest value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] partition.assignment.strategy = [org.apache.kafka.clients.consumer.RangeAssignor] ssl.endpoint.identification.algorithm = null max.partition.fetch.bytes = 2097152 ssl.keystore.location = null ssl.truststore.location = null ssl.keystore.password = null metrics.sample.window.ms = 30000 metadata.max.age.ms = 300000 security.protocol = PLAINTEXT auto.commit.interval.ms = 5000 ssl.protocol = TLS sasl.kerberos.min.time.before.relogin = 60000 connections.max.idle.ms = 540000 ssl.trustmanager.algorithm = PKIX group.id = test7 enable.auto.commit = false metric.reporters = [] ssl.truststore.type = JKS send.buffer.bytes = 131072 reconnect.backoff.ms = 50 metrics.num.samples = 2 ssl.keystore.type = JKS heartbeat.interval.ms = 3000 

Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:624) at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:518) at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:500)

Answer 1

These conditions needed to be keep in mind to change session.timeout.ms:

1. group.max.session.timeout.ms in the server.properties > session.timeout.ms in the consumer.properties.
2. group.min.session.timeout.ms in the server.properties < session.timeout.ms in the consumer.properties.
3. request.timeout.ms > session.timeout.ms + fetch.wait.max.ms
4. (session.timeout.ms)/3 > heartbeat.interval.ms
5. session.timeout.ms > Worst case processing time of Consumer Records per consumer poll(ms).

Answer 2

The range of consumer session timeout is controlled by broker group.max.session.timeout.ms(default 30s) and group.min.session.timeout.ms(default 6s).

You should increase group.max.session.timeout.ms first in broker side, otherwise you will get "The session timeout is not within an acceptable range.".