What is the usage of Configured class in Hadoop programs?

Question

Most of Hadoop MapReduce programs are like this:

public class MyApp extends Configured Implements Tool {
    @Override
    public int run(String[] args) throws Exception {
        Job job = new Job(getConf());
        /* process command line options */
        return job.waitForCompletion(true) ? 0 : 1;
    }
    public static void main(String[] args) throws Exception {
        int exitCode = ToolRunner.run(new MyApp(), args);
        System.exit(exitCode);
    }
}

What is the usage of Configured? As Tool and Configured both have getConf() and setConf() in common. What does it provide to our application?

Answer 1

Configured is an implementation class of the interface Configurable. Configured is the base class which has the implementations of getConf() and setConf().

Merely extending this base class enables the class that extends this to be configured using a Configuration and there are more than one implementations for Configuration.

When your code executes the following line,

ToolRunner.run(new MyApp(), args);

Internally it will do this

ToolRunner.run(tool.getConf(), tool, args);

In the above case tool is the MyApp class instance which is an implementation of Tool which just as you said has getConf() but it is just as an interface. The implementation is coming from Configured base class. If you avoid extending Configured class in the above code, then you will have to do the getConf() and setConf() implementations on your own.