Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

what is the basic difference between jobconf and job?

hi i wanted to know the basic difference between jobconf and job objects,currently i am submitting my job like this

JobClient.runJob(jobconf);

i saw other way of submitting jobs like this

Configuration conf = getConf();
Job job = new Job(conf, "secondary sort");
job.waitForCompletion(true);     
return 0;

and how can i specify the sortcomparator class for the job using jobconf?

can any one explain me this concept?

like image 652
user1585111 Avatar asked Aug 23 '13 12:08

user1585111


People also ask

What is JobConf in Hadoop?

JobConf is the primary interface for a user to describe a map-reduce job to the Hadoop framework for execution. The framework tries to faithfully execute the job as-is described by JobConf , however: Some configuration parameters might have been marked as final by administrators and hence cannot be altered.

How to Configure a job in hadoop?

Here is an example on how to configure a job via JobConf : // Create a new JobConf JobConf job = new JobConf(new Configuration(), MyJob. class); // Specify various job-specific parameters job. setJobName("myjob"); FileInputFormat.


1 Answers

In short: JobConf and everything else in the org.apache.hadoop.mapred package is part of the old API used to write hadoop jobs, Job and everything in the org.apache.hadoop.mapreduce package is part of the new and preferred API to write hadoop jobs. Both APIs generally provide equivalent core functionality.

If you're new to hadoop just start using the new API (i.e. Job and Configuration instead of JobConf). Make sure to not import anything from the mapred package. When you find examples on the internet using the old API you can use this presentation or this guide to translate it to the new API.

like image 61
jkovacs Avatar answered Oct 06 '22 00:10

jkovacs