zookeeper client does not provide CLI with "jline support is disabled" message

Question

I just brought up CDH 5.4 and installed zookeeper. I used zkCli successfully many times before. This time the command line launch stops before getting to the prompt with

Welcome to ZooKeeper!
JLine support is disabled
2015-05-04 18:18:33,936 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@975] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2015-05-04 18:18:33,952 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@852] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2015-05-04 18:18:33,985 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x34d12349d0a15cf, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

I know the usual print out is JLine support is enabled

Is that what's getting it stuck? I don't see any way to change this one the Cloudera manager config page.

Answer 1

Disclaimer: please note answer is from 2015. It is probably not relevant anymore but may serve historical purpose and have some light on similar 'modern' cases.

SHORT

Cloudera has broken zookeeper 3.4.5-cdh5.4.0 in several places. Service is working but CLI is dead. No workaround other than roll back.

LONG

Assign a bounty on this ;-). I have stepped on this mine too and was angry enough to find the reason:

Zookeeper checks JLine during ZooKeeperMain.run(). There is try-catch block which loads number of classes. Any exception during class loading fails whole block and JLine support is reported to be disabled.

But here is why this happens with CDH 5.4.0:

1. Current opensource Zookeeper-3.4.6 works against jline-0.9.94. Has no such issue.

2. In CDH 5.4 Cloudera has applied the following patch:

roman@node4:$ diff zookeeper-3.4.5-cdh5.3.3/src/java/main/org/apache/zookeeper/ZooKeeperMain.java zookeeper-3.4.5-cdh5.4.0/src/java/main/org/apache/zookeeper/ZooKeeperMain.java

305,306c305,306
<                 Class consoleC = Class.forName("jline.ConsoleReader");
<                 Class completorC =
---
>                 Class consoleC = Class.forName("jline.ConsoleReader");
>                 Class completorC =
316,317c316,317
<                 Method addCompletor = consoleC.getMethod("addCompletor",
<                         Class.forName("jline.Completor"));
---
>                 Method addCompletor = consoleC.getMethod("addCompleter",
>                         Class.forName("jline.console.completer.Completer"));

3. CDH 5.4 uses jline-2.11.jar for ZooKeeper and it has no jline.ConsoleReader class (from 2.11 it is jline.console.ConsoleReader).

4. Jline 0.9.94 in turn has no jline.console.completer.Completer.

So there is incompatibility with any existing JLine. Any Cloudera CDH 5.4 user can run zookeeper-client on his/her cluster and find it does not work.

Open source zookeeper-3.4.6 depends on jline-0.9.94 which has no such patches. Don't know why Cloudera engineers have done such a mine.

I see no clean way to fix it with 3.4.5-cdh5.4.0. I stayed with 3.4.5-cdh5.3.3 dependency where I need CLI and have production clusters.

5. It seemed to me both jline-0.9.94.jar and jline.2.11.jar in classpath for zookeeper will fix the problem. But just have found Cloudera made another 'fix' in ZK for CDH 5.4.0, they have renamed org.apache.zookeeper.JLineZNodeCompletor class to org.apache.zookeeper.JLineZNodeCompleter.

But here is the code from ZooKeeperMain.java

Class<?> completorC =                    Class.forName("org.apache.zookeeper.JLineZNodeCompletor");

And of course it meaan practically it is not possible to start ZK CLI in CDH 5.4.0 proper way. Awful work. :-(