Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use map/reduce to handle more than 10000 unique keys for grouping in MongoDB?

Tags:

mongodb

I am using MongoDB v1.4 and the mongodb-csharp driver and I try to group on a data store that has more than 10000 keys, so I get this error:

assertion: group() can't handle more than 10000 unique keys

using c# code like this:

Document query = new Document().Append("group",
new Document()
.Append("key", new Document().Append("myfieldname", true).Append("length", true))
.Append("$reduce",
      new CodeWScope(
          "function(obj,prev) { prev.count++; }"))
.Append("initial", new Document().Append("count", 0))
.Append("ns", "myitems"));

I read that I should use map/reduce, but I can't figure out how. Can somebody please shed some light on how to use map/reduce?
Or is there any other way to get around this limitation?
Thanks.

EDIT: I forgot that I have 2 columns in my key collection, added that.

like image 235
Magnus Johansson Avatar asked Oct 14 '22 07:10

Magnus Johansson


1 Answers

Thanks to Darin Dimitrov.

In addition, I will post my solution that group by two fields, if anybody is interested in that:

string mapFunction = @"
  function(){
    emit({
      fieldname:this.fieldname, 
      length:this.length
    }, 1)
  }";

string reduceFunction =
@"function(k,vals)          
      {
       var sum = 0;
        for(var i in vals) {
          sum += vals[i];
        }
        return sum;
      }";

IMongoCollection mrCol = db["table"];

using (MapReduceBuilder mrb = mrCol.MapReduceBuilder().Map(mapFunction).Reduce(reduceFunction))
{
  using (MapReduce mr = mrb.Execute())
  {
    foreach (Document doc in mr.Documents)
    {
      // do something
      int groupCount = Convert.ToInt32(doc["value"]);

      string fieldName = ((Document)doc["_id"])["fieldname"].ToString();
    }
  }
}
like image 193
Magnus Johansson Avatar answered Oct 18 '22 14:10

Magnus Johansson