Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Run a Hadoop job without output file

Tags:

hadoop

Is it possible to run a hadoop job without specifying output file ? When i try to run a hadoop job , no output file specified Exception is thrown . can any one please give any procedure to do so using Java.

I am writing the data processed by reduce to a non relational database so i no longer require it to write to HDFS.

like image 445
R45c4l Avatar asked Oct 03 '12 11:10

R45c4l


1 Answers

Unfortunately, you can't really do this. Writing output is part of the framework. When you work outside of the framework, you basically have to just deal with the consequences.

You can use NullOutputFormat, which doesn't write any data to HDFS. I think it still creates the folder, though. You could always let Hadoop create the folder, then delete it.

like image 69
Donald Miner Avatar answered Nov 29 '22 03:11

Donald Miner