Failed to detect a valid hadoop home directory

Question

I have set up Hadoop 2.2.0 single node and started it up. I am able to browse the FS on http://localhost:50070/ Then I tried to write a dummy file using the following code.

public class Test {
public void write(File file) throws IOException{
    FileSystem fs = FileSystem.get(new Configuration());
    Path outFile = new Path("test.jpg");       
    FSDataOutputStream out = fs.create(outFile);        

}    

I get the following exception

INFO:   DEBUG - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
    INFO:   DEBUG - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
    INFO:   DEBUG - UgiMetrics, User and group related metrics
    INFO:   DEBUG -  Creating new Groups object
    INFO:   DEBUG - Trying to load the custom-built native-hadoop library...
    INFO:   DEBUG - Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
    INFO:   DEBUG - java.library.path=/usr/lib/jvm/jdk1.7.0/jre/lib/amd64:/usr/lib/jvm/jdk1.7.0/jre/lib/i386::/usr/java/packages/lib/i386:/lib:/usr/lib
    INFO:   WARN - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    INFO:   DEBUG - Falling back to shell based
    INFO:   DEBUG - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
    INFO:   DEBUG - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000
    INFO:   DEBUG - hadoop login
    INFO:   DEBUG - hadoop login commit
    INFO:   DEBUG - using local user:UnixPrincipal: qualebs
    INFO:   DEBUG - UGI loginUser:qualebs (auth:SIMPLE)
    INFO:   DEBUG - Failed to detect a valid hadoop home directory
    java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
        at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:225)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:250)
        at 
    org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
        at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:468)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:905)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:886)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:783)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:772)
        at com.qualebs.managers.HadoopDFS.writer(HadoopDFS.java:41)

Where do I set the HADOOP_HOME or hadoop.home.dir? Operating system is Ubuntu 11.10

The only configuration files I have configured are the following with the properties added

1. core-site.xml

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

2. hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

3. mapred-site.xml.template

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:9001</value>
    </property>
</configuration>

Eagerly awaiting your responses.

Answer 1

I found my solution by doing this:

System.setProperty("hadoop.home.dir", "/");

This exception is thrown by checkHadoopHome() in org.apache.hadoop.util.Shell

Hope it helps!