Consuming from a replica

Question

Kafka replicates each partition of a topic up-to specified replication factor.

As far as I know, all write and read requests are routed to the leader of the partition. Is there any way to consume from the followers not from the leader?

Is the replication in Kafka only for fail-over?

Answer 1

In Kafka 2.3 and older, you can only consume from the leader -- this is by design. Replication is for fault-tolerance only.

If the leader fails, one of the follower will be elected as a new leader.

Have a look at this blog post for more details: http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/

Update:

As of Kafka 2.4, fetching from follower replicas is possible. For details, check out KIP-392: https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica