Write a file in hdfs with Java

Question

I want to create a file in HDFS and write data in that. I used this code:

Configuration config = new Configuration();      FileSystem fs = FileSystem.get(config);  Path filenamePath = new Path("input.txt");   try {     if (fs.exists(filenamePath)) {         fs.delete(filenamePath, true);     }      FSDataOutputStream fin = fs.create(filenamePath);     fin.writeUTF("hello");     fin.close(); } 

It creates the file, but it does't write anything in it. I searched a lot but didn't find anything. What is my problem? Do I need any permission to write in HDFS?

Thanks.

Answer 1

an alternative to @Tariq's asnwer you could pass the URI when getting the filesystem

import org.apache.hadoop.fs.FileSystem import org.apache.hadoop.conf.Configuration import java.net.URI import org.apache.hadoop.fs.Path import org.apache.hadoop.util.Progressable import java.io.BufferedWriter import java.io.OutputStreamWriter  Configuration configuration = new Configuration(); FileSystem hdfs = FileSystem.get( new URI( "hdfs://localhost:54310" ), configuration ); Path file = new Path("hdfs://localhost:54310/s2013/batch/table.html"); if ( hdfs.exists( file )) { hdfs.delete( file, true ); }  OutputStream os = hdfs.create( file,     new Progressable() {         public void progress() {             out.println("...bytes written: [ "+bytesWritten+" ]");         } }); BufferedWriter br = new BufferedWriter( new OutputStreamWriter( os, "UTF-8" ) ); br.write("Hello World"); br.close(); hdfs.close(); 

Answer 2

Either define the HADOOP_CONF_DIR environment variable to your Hadoop configuration folder or add the following 2 lines in your code :

config.addResource(new Path("/HADOOP_HOME/conf/core-site.xml")); config.addResource(new Path("/HADOOP_HOME/conf/hdfs-site.xml")); 

If you don't add this, your client will try to write to the local FS, hence resulting into the permission denied exception.