Menu
When I execute the order
"bin/hadoop namenode -format"
in Linux, I got the below warning,
"WARN common.Util: Path /data/dfs/name should be specified as a URI in configuration files. Please update hdfs configuration."
the namenode dir setting in the file hdfs-site.xml is
<property>
<name>dfs.namenode.name.dir</name>
<value>/data/dfs/name</value>
<final>true</final>
</property>
when I changed it to
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///data/dfs/name</value>
<final>true</final>
</property>
the warning disappeared, so what is the meaning of "file://", why should we add it there?
879
Hadoop NameNode is the centralized place of an HDFS file system which keeps the directory tree of all files in the file system, and it tracks where across the cluster the file data is kept. In short, it keeps the metadata related to datanodes.
When the NameNode goes down, the file system goes offline. There is an optional SecondaryNameNode that can be hosted on a separate machine. It only creates checkpoints of the namespace by merging the edits file into the fsimage file and does not provide any real redundancy.
Safemode for the NameNode is essentially a read-only mode for the Hadoop Distributed File System (HDFS) cluster. NameNode might enter into Safemode for different reasons, such as the following: Available space is less than the amount of space required for the NameNode storage directory.
NameNode is the master node in the Apache Hadoop HDFS Architecture that maintains and manages the blocks present on the DataNodes (slave nodes). NameNode is a very highly available server that manages the File System Namespace and controls access to files by clients.
It's a major bug https://issues.apache.org/jira/browse/HADOOP-15772 and fixed in this commit https://github.com/apache/hadoop/commit/2eb597b1511f8f46866abe4eeec820f4191cc295
You don't need to worry if you hit this issue/bug. It's perfectly fine and ignore this warning.
The description goes like this.
The following warnings are logged on service startup and are noise. It is perfectly valid to list local paths without using the URI syntax.
2018-09-16 23:16:11,393 WARN common.Util (Util.java:stringAsURI(99)) - Path /hadoop/hdfs/namenode should be specified as a URI in configuration files. Please update hdfs configuration.
Also, Log level has changed from WARNING to INFO with this message
Assuming 'file' scheme for path /hadoop/hdfs/namenode in configuration.
130
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
