Map and Reduce in .NET

Question

What scenarios would warrant the use of the "Map and Reduce" algorithm?

Is there a .NET implementation of this algorithm?

Answer 1

Linq equivalents of Map and Reduce: If you’re lucky enough to have linq then you don’t need to write your own map and reduce functions. C# 3.5 and Linq already has it albeit under different names.

• Map is Select:

  Enumerable.Range(1, 10).Select(x => x + 2);
  

• Reduce is Aggregate:

  Enumerable.Range(1, 10).Aggregate(0, (acc, x) => acc + x);
  

• Filter is Where:

  Enumerable.Range(1, 10).Where(x => x % 2 == 0);
  

https://www.justinshield.com/2011/06/mapreduce-in-c/

Answer 2

The classes of problem that are well suited for a mapreduce style solution are problems of aggregation. Of extracting data from a dataset. In C#, one could take advantage of LINQ to program in this style.

From the following article: http://codecube.net/2009/02/mapreduce-in-c-using-linq/

the GroupBy method is acting as the map, while the Select method does the job of reducing the intermediate results into the final list of results.

var wordOccurrences = words
                .GroupBy(w => w)
                .Select(intermediate => new
                {
                    Word = intermediate.Key,
                    Frequency = intermediate.Sum(w => 1)
                })
                .Where(w => w.Frequency > 10)
                .OrderBy(w => w.Frequency);

For the distributed portion, you could check out DryadLINQ: http://research.microsoft.com/en-us/projects/dryadlinq/default.aspx