Where is yarn.log.dir defined?

Question

In yarn-default.xml for Apache Hadoop 3.0.0 it shows the default value for yarn.nodemanager.log-dirs as ${yarn.log.dir}/userlogs.

Where is yarn.log.dir defined? Does it have a default value?

I do not find it in any of the default configurations (core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml).

I do not find it mentioned in any of the environment scripts (hadoop-env.sh,httpfs-env.sh,kms-env.sh,mapred-env.sh,yarn-env.sh.

Equally baffling to me is that when I grep the code for "yarn.nodemanager.log-dirs" the only places it shows up are in yarn-default.xml and markdown files, not in any java code anywhere. So how does setting yarn.nodemanager.log-dirs do anything?

Answer 1

yarn.log.dir is a Java property, based by a -D flag

In the yarn-env.sh, you should see YARN_LOG_DIR

# default log directory and file
if [ "$YARN_LOG_DIR" = "" ]; then
  YARN_LOG_DIR="$HADOOP_YARN_HOME/logs"
  ...
fi

YARN_OPTS="$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR"