Chaining Hadoop MapReduce 1.1.1 Example

Question

I'm very much a Hadoop beginner, but in Hadoop 1.1.1, I'm trying to do map -> reduce -> map -> reduce. I'm having a difficult time finding a concrete example of how to do this in Java. I'm really curious what the whole driver method would look like. I'm currently using the org.apache.hadoop.mapreduce libraries (which I believe is the newest codebase).

Also, I'd prefer not to install another library like Oozie if I don't have to.

Answer 1

I think this may help you: http://blogs.msdn.com/b/avkashchauhan/archive/2012/03/29/how-to-chain-multiple-mapreduce-jobs-in-hadoop.aspx

You may also look into ChainMapper(http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/ChainMapper.html) depending on what you are trying to achieve. This works only with one reduce, though. If you want to go specifically map -> reduce -> map -> reduce, I would go with my first link.