In which folder or where actually the fsimage and edit log files are stored for the namenode to read and merge during the startup?

Question

When a namenode starts up, it reads HDFS state from an image file, fsimage and then applies the edits from the edit log file.

If I am not wrong , the Name node starts up means when we write start-all.sh. So during this start up time I think it read the fsimage and edit logs and merge them . But from which folder or from which location it actually read both these things?

Answer 1

In hadoop-1.x start-all.sh script internally performs two operation start-dfs.sh and start-mapred.sh. start-dfs.sh will start all daemons required for hdfs ie : datanode, namenode, secondary namenode

The checkpoint operation(applying edit logs to fsimage) happens during namenode start and this activity can be configured during namenode runs by tuning the parameter hdfs-site.xml --> dfs.namenode.checkpoint.period .

During namenode starts namenode daemons loads fsimage from the directory specified in hdfs-site.xml -> dfs.name.dir. This property should have been overridden otherwise it would take the default value (file:///tmp/dfs/name/)

Location of the edit logs can be found by checking the value of hdfs-site.xml -> dfs.name.edits.dir. default value of dfs.name.edits.dir is ${dfs.name.dir}.

The above property names are changed in hadoop-2.0