What does "Broker transport failure" mean in kafka?

Question

I am using a python client(Confulent kafka) to consume from kafka. Some times the consumer will hit error like below:

ERROR KafkaError{code=_TRANSPORT,val=-195,str="GroupCoordinator response error: Local: Broker transport failure"}

Can someone help to explain what does the error mean? "transport failure" seems mean the consumer is having network issue with the broker, is that right? what should I do when this error happen?

Answer 1

The question summary asks about "Broker transport failure", which could be many different things. But focusing on the the question details which give the full error message as "GroupCoordinator response error: Local: Broker transport failure", this could well be caused by:

https://github.com/edenhill/librdkafka/issues/1987

If so, then it's not really an error and can be safely ignored. The Kafka client will handle it, though it's annoying everyone gets this error message in their logs and then needs to go investigate it

The proposed explanation given on the github issue above is:

there may be a case when the idle connection reaper may "kill" a broker connection at exactly the same time a consumer process makes a GroupCoordinator request to it. In that case the poll method will return the error message to the application (let's say similar to the PARTITION_EOF message).

Even if that explantion is not 100% correct, one of the maintainers says in the same thread:

As a general rule for librdkafka-based clients: given that the cluster and client are correctly configured, all errors can be ignored as they are most likely temporary and librdkafka will attempt to recover automatically.