How to convert .txt file to Hadoop's sequence file format

Question

To effectively utilise map-reduce jobs in Hadoop, i need data to be stored in hadoop's sequence file format. However,currently the data is only in flat .txt format.Can anyone suggest a way i can convert a .txt file to a sequence file?

Answer 1

So the way more simplest answer is just an "identity" job that has a SequenceFile output.

Looks like this in java:

    public static void main(String[] args) throws IOException,         InterruptedException, ClassNotFoundException {      Configuration conf = new Configuration();     Job job = new Job(conf);     job.setJobName("Convert Text");     job.setJarByClass(Mapper.class);      job.setMapperClass(Mapper.class);     job.setReducerClass(Reducer.class);      // increase if you need sorting or a special number of files     job.setNumReduceTasks(0);      job.setOutputKeyClass(LongWritable.class);     job.setOutputValueClass(Text.class);      job.setOutputFormatClass(SequenceFileOutputFormat.class);     job.setInputFormatClass(TextInputFormat.class);      TextInputFormat.addInputPath(job, new Path("/lol"));     SequenceFileOutputFormat.setOutputPath(job, new Path("/lolz"));      // submit and wait for completion     job.waitForCompletion(true);    }