Is it possible to run zeppelin with spark yarn-cluster

Question

It runs well with spark yarn-client but what about yarn-cluster? If it's possible then please share how we can do that.

Answer 1

Yes, it is possible in current master version of Zeppelin.

Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235

Yarn Cluster Mode for Spark Interpreter

Before 0.8.0, Zeppelin only supported yarn client mode for Spark Interpreter which means the driver would run in the same host of Zeppelin Server. The incur high memory pressure of the Zeppelin Server host especially when you run Spark Interpreter in isolated mode.

https://issues.apache.org/jira/browse/ZEPPELIN-2898 (merged to master end of Sep/2017).

Update: Notice 0.8.0 is released which has this feature in.