Reading HDFS and local files in Java

Question

I want to read file paths irrespective of whether they are HDFS or local. Currently, I pass the local paths with the prefix file:// and HDFS paths with the prefix hdfs:// and write some code as the following

Configuration configuration = new Configuration(); FileSystem fileSystem = null; if (filePath.startsWith("hdfs://")) {   fileSystem = FileSystem.get(configuration); } else if (filePath.startsWith("file://")) {   fileSystem = FileSystem.getLocal(configuration).getRawFileSystem(); } 

From here I use the API's of the FileSystem to read the file.

Can you please let me know if there is any other better way than this?

Answer 1

Does this make sense,

public static void main(String[] args) throws IOException {      Configuration conf = new Configuration();     conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));     conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/hdfs-site.xml"));      BufferedReader br = new BufferedReader(new InputStreamReader(System.in));     System.out.println("Enter the file path...");     String filePath = br.readLine();      Path path = new Path(filePath);     FileSystem fs = path.getFileSystem(conf);     FSDataInputStream inputStream = fs.open(path);     System.out.println(inputStream.available());     fs.close(); } 

You don't have to put that check if you go this way. Get the FileSystem directly from Path and then do whatever you feel like.