How to specify Spark properties when starting Spark History Server?

Question

Does anyone know how to set values in the SparkConf when starting the Spark History Server?

Answer 1

if you are using <SPARK_HOME>/sbin/start-history-server.sh then you cannot specify command line argument but you can specify SPARK_HISTORY_OPTS as environment variable and specify the various environment variables like: -

export SPARK_HISTORY_OPTS="$SPARK_HISTORY_OPTS -Dspark.history.ui.port=9000

but if you are using <SPARK_HOME>/sbin/start-daemon.sh script then you can specify multiple command line options. like this: -

<SPARK_HOME>/sbin/spark-daemon.sh start org.apache.spark.deploy.history.HistoryServer -Dspark.history.ui.port=9000