Spark-submit how to set the user.name

Question

want to set

mapreduce.job.user.name=myuser

Tried

spark-submit  --class com.MyClass
--conf mapreduce.job.user.name=myuser \ 
--conf spark.mapreduce.job.user.name=myuser \
--master yarn  \
--deploy-mode cluster \

Also tried

--conf user.name

in environment of Spark UI showing

user.name yarn

Answer 1

In Spark 3, set SPARK_USER as a system property.

https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/Utils.scala

  /**
   * Returns the current user name. This is the currently logged in user, unless that's been
   * overridden by the `SPARK_USER` environment variable.
   */
  def getCurrentUserName(): String = {
    Option(System.getenv("SPARK_USER"))
      .getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
  }