Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mongodb mapreduce iterate through object's key value pairs

I have a mongodb collection that has the following data :

{ "_id" : ObjectId("4da31b8b5ba19e3c11345a66"), "USERID" : 4, "datekey" : "BIBAK", "balancekey" : "MAIYM" }
{ "_id" : ObjectId("4da31b8b5ba19e3c12345a66"), "USERID" : 4, "datekey" : "QOTWH", "balancekey" : "SFEYQ" }
{ "_id" : ObjectId("4da31b8b5ba19e3c14345a66"), "USERID" : 4, "datekey" : "TLWJJ", "balancekey" : "RDKAM" }
{ "_id" : ObjectId("4da31b8b5ba19e3c15345a66"), "USERID" : 5, "emailadress" : "KBDIJD" }
{ "_id" : ObjectId("4da31b8b5ba19e3c16345a66"), "USERID" : 1, "accountname" : "KL", "weblink" : "GJ", "note" : "KP" }
{ "_id" : ObjectId("4da31b8b5ba19e3c17345a66"), "USERID" : 1, "accountname" : "WD", "weblink" : "SZ", "note" : "CL" }
{ "_id" : ObjectId("4da31b8b5ba19e3c18345a66"), "USERID" : 1, "accountname" : "IK", "weblink" : "OK", "note" : "HD" }
{ "_id" : ObjectId("4da31b8b5ba19e3c19345a66"), "USERID" : 4, "datekey" : "UGBYH", "balancekey" : "VOPRX" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1a345a66"), "USERID" : 3, "userid" : "ZBWD", "password" : "FZAK", "key" : "QMEE" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1b345a66"), "USERID" : 1, "accountname" : "GH", "weblink" : "MY", "note" : "QU" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1c345a66"), "USERID" : 3, "userid" : "YZMW", "password" : "MVUR", "key" : "YSZC" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1d345a66"), "USERID" : 4, "datekey" : "LIEWF", "balancekey" : "THXYR" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1e345a66"), "USERID" : 4, "datekey" : "UIWOY", "balancekey" : "SKOKG" }
{ "_id" : ObjectId("4da31b8b5ba19e3c1f345a66"), "USERID" : 4, "datekey" : "POYKK", "balancekey" : "KZGDZ" }
{ "_id" : ObjectId("4da31b8b5ba19e3c20345a66"), "USERID" : 4, "datekey" : "LWNXW", "balancekey" : "VJXFC" }
{ "_id" : ObjectId("4da31b8b5ba19e3c23345a66"), "USERID" : 4, "datekey" : "IYMGO", "balancekey" : "RWBUE" }
{ "_id" : ObjectId("4da31b8b5ba19e3c24345a66"), "USERID" : 3, "userid" : "CJTH", "password" : "YQCL", "key" : "PCDB" }
{ "_id" : ObjectId("4da31b8b5ba19e3c25345a66"), "USERID" : 4, "datekey" : "OBOCN", "balancekey" : "XOHWA" }
{ "_id" : ObjectId("4da31b8b5ba19e3c26345a66"), "USERID" : 3, "userid" : "EHTQ", "password" : "KBXV", "key" : "YAMD" }
{ "_id" : ObjectId("4da31b8b5ba19e3c27345a66"), "USERID" : 5, "emailadress" : "VYSAHK" }

I need help writing a mapreduce functions to generate a string that would be like a csv structure.. so for example if i need data for userid = 4, the result would be

datekey,balancekey\n
BIBAK,MAIYM\n
QOTWH,SFEYQ\n
......

I am facing a problem doing the following .. Since each userid has different data, I need a way to go through those key/value pairs in a generic way .. So pretty much the question is how to loop through the objects parameters and get their values in order to emit them . and then in the reduce function i can concatenate them and add the \n.

Thanks

like image 429
Wael Awada Avatar asked Mar 16 '26 11:03

Wael Awada


1 Answers

You can have csv values for each document using following code. There are two variants of map() function provided as example. First one is non-generic and helps understand the concept. While one, at the end, is generic. Try running both of them and you will understand the difference.

Map function - Not Generic

var map = function() {
      var outputString =  "";
      if (this.datekey){
           outputString += this.datekey;
      }  
      outputString += "," ;
      if (this.balancekey){
           outputString += this.balancekey;
      }  
      emit(this.USERID, {outputString:outputString});
};

Reduce function

var reduce = function(key,values){
    overallOutputString = "";
    values.forEach(function(i){
        overallOutputString += i.outputString+"\n";
    });
    return { outputString:overallOutputString};
};

Performing the M/R operation

var result  = db.items.mapReduce( map,
                                  reduce,
                                  {query:{USERID:{$exists:true}},
                                   out: {replace:"csv_dump"}
                                  });

Observing the output

db.csv_dump.find();

Map function - Generic

var map = function() {
      var outputString =  "";
      for(var prop in this ) {
        if (prop != "_id" && prop != "USERID" ){
            outputString += this[prop]+",";
        }
      }
      outputString = outputString.substring(0, outputString.length-1); // removing last comma
      emit(this.USERID, {outputString:outputString});
};
like image 145
Samyak Bhuta Avatar answered Mar 19 '26 01:03

Samyak Bhuta



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!