Menu
I'd like to find out good and robust MapReduce framework, to be utilized from Scala.
531
Write an engine to execute MapReduce programs. It can be written in either Python, Java or Scala.
Scala | reduce() Function The reduce() method is a higher-order function that takes all the elements in a collection (Array, List, etc) and combines them using a binary operation to produce a single value. It is necessary to make sure that operations are commutative and associative.
There are two phases in the MapReduce program, Map and Reduce. The Map task includes splitting and mapping of the data by taking a dataset and converting it into another set of data, where the individual elements get broken down into tuples i.e. key/value pairs.
To add to the answer on Hadoop: there are at least two Scala wrappers that make working with Hadoop more palatable.
Scala Map Reduce (SMR): http://scala-blogs.org/2008/09/scalable-language-and-scalable.html
SHadoop: http://jonhnny-weslley.blogspot.com/2008/05/shadoop.html
UPD 5 oct. 11
There is also Scoobi framework, that has awesome expressiveness.
157
http://hadoop.apache.org/ is language agnostic.
42
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
