Kafka Consumer startup error: Failed to add leader for partitions [calls,0] - NotLeaderForPartitionException

Question

Note: this is for an older version : We are running kafka_2.9.2-0.8.1

When attempting to run the kafka-console-consumer.bat against an existing topic on windows7 we receive "failed to add leader" and "NotLeaderForPartition" exceptions

Here is the command line

set GROUP=group1234
kafka-console-consumer.bat --group %GROUP%  --zookeeper localhost:2181 --topic calls --from-beginning

Here are the errors:

[2014-05-26 15:02:12,997] WARN [group1234_S80035683-SC01-1401141732400-98745e28-leader-finder-thread], Failed to add leader for partitions [calls,0];
will retry (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread:89)
kafka.common.NotLeaderForPartitionException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at java.lang.Class.newInstance(Class.java:374)
        at kafka.common.ErrorMapping$.exceptionFor(ErrorMapping.scala:73)
        at kafka.consumer.SimpleConsumer.earliestOrLatestOffset(SimpleConsumer.scala:160)
        at kafka.consumer.ConsumerFetcherThread.handleOffsetOutOfRange(ConsumerFetcherThread.scala:60)
        at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:179)
        at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:174)
        at scala.collection.immutable.Map$Map1.foreach(Map.scala:119)
        at kafka.server.AbstractFetcherThread.addPartitions(AbstractFetcherThread.scala:174)
        at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:86)
        at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:76)
        at scala.collection.immutable.Map$Map1.foreach(Map.scala:119)
        at kafka.server.AbstractFetcherManager.addFetcherForPartitions(AbstractFetcherManager.scala:76)

And we are unable to consume any messages in the end.

We are running kafka_2.9.2-0.8.1

Note: Zookeeper is seeing the Consumer attempt to connect: I go into zkCli and can see the new group1234

[zk: localhost:2181(CONNECTED) 2] ls2 /consumers/group1234
[owners, ids]
cZxid = 0x123
ctime = Mon May 26 15:02:12 PDT 2014
mZxid = 0x123
mtime = Mon May 26 15:02:12 PDT 2014
pZxid = 0x128
cversion = 2
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 2

And here is info on the requested calls topic in ZK:

[zk: localhost:2181(CONNECTED) 7] ls2 /brokers/topics/calls
[partitions]
cZxid = 0x18
ctime = Sat May 24 23:15:16 PDT 2014
mZxid = 0x18
mtime = Sat May 24 23:15:16 PDT 2014
pZxid = 0x1c
cversion = 1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 36
numChildren = 1

In case there were corruption on the topic, I just dropped it in ZK and then recreated it via kafka-topics.bat. Here is the new ZK output

[zk: localhost:2181(CONNECTED) 15] ls2 /brokers/topics/calls
[]
cZxid = 0x136
ctime = Mon May 26 16:02:51 PDT 2014
mZxid = 0x136
mtime = Mon May 26 16:02:51 PDT 2014
pZxid = 0x136
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 36
numChildren = 0

Answer 1

A search shows that now, 7 years later, this is no longer a known problem for current versions.

There have also been multiple patches which resolve errors that may or may not be the same, and it is almost certain that one of these fixed the issue.

As such the only practical solution for anyone seems to be upgrading to a newer version. (For Kafka, Zookeeper as well as Windows.)

If the problem persists in currently relevant versions please ask a new question as the root cause is unlikely to be the same.