MongoDB Aggregate lookup in Go (mgo.v2)

Question

I'm trying to implement $lookup functionality in one of my mongoDB queries in go (golang) using the mgo package.

Below are my collections:

folders:

"_id"    : ObjectId("22222222222222"),
"name"   : "Media",
"level"  : 1,
"userIDs": [ObjectId("4444444444444")]

documents:

"_id"      : ObjectId("11111111111111"),
"title"    : "Media Management",
"body"     : BinData(0,"PvQ6z2NBm4265duo/e2XsYxA5bXKo="),
"level"    : 1,
"folderID" : ObjectId("22222222222222"), // Foreign Key/Field
"userIDs"  : [ObjectId("44444444444444")]

Below is the query I've written that successfully runs on the shell:

var query = [
{ 
  "$lookup": {
    "from":         "documents",
    "localField":   "_id",
    "foreignField": "folderID",
    "as":           "documents",
  }
}
 ,{
   "$match": {
      "userIDs": ObjectId("userIdHere"), // filder by a userID
      "level": {$gte: 0},                // filter by a folder level
    },
  }
];

db.folders.aggregate(query).pretty().shellPrint();

If I run this script on the shell, I get the desired result. Basically, the folder collection is returned to me containing the full relevant documents that were linked through the $lookup. I'm not including it here because this question already seems too long.

I've tried to translate this query into something that mgo would be able to parse and execute. Here it is below in go code:

query := bson.M{
  "$lookup": bson.M{ // lookup the documents table here
  "from":         "documents",
  "localField":   "_id",
  "foreignField": "folderID",
  "as":           "documents",
},
  "$match": bson.M{
    "level":   bson.M{"$gte": user.Level}, // filter by level
    "userIDs": user.ID,                    // filter by user
  },
}

pipe := collection.Pipe(query) // querying the "folders" collection
err := pipe.All(&result)

I always get the same error: wrong type for field (pipeline) 3 != 4

If I understand correctly, it's because it can't properly parse the result back into the $result object. I've done everything I can to ensure the struct has the exact structure that is required. I've also tried to pass in a genereric []interface{} and an empty bson.M{} objects. Still receive the same error.

Below is my Folders struct:

type Folder struct {
  ID        bson.ObjectId   `json:"id" bson:"_id"`
  Name      string          `json:"name"`
  Level     int             `json:"level"`
  UserIDs   []bson.ObjectId `json:"userIDs" bson:"userIDs"`
  Users     []User          `json:"-" bson:"-"` // doesn't get stored in the database
  Documents []Document      `json:"-" bson:"-"` // doesn't get stored in the database
}

I've also removed the $match clause to see if I could get anything at all back from that $lookup query. But I still get the same error.

Perhaps the mgo package doesn't support $lookup? If so, would there be another way? Perhaps I could send the raw query text to mongo and receive the raw response and parse it myself?

Answer 1

Found the solution!

The trick was to create the query in a slice ([]bson.M) and change the structure of the query a bit:

query := []bson.M{{
  "$lookup": bson.M{ // lookup the documents table here
    "from":         "documents",
    "localField":   "_id",
    "foreignField": "folderID",
    "as":           "documents",
  }},
  {"$match": bson.M{
    "level": bson.M{"$lte": user.Level},
    "userIDs": user.ID,
}}}

pipe := collection.Pipe(query)
err := pipe.All(&folders)

I found a clue in mgo's Pipe function docs. Also, I had to change the tags for the Documents field in my Folders struct for mgo to pupolate that field:

type Folder struct {
  ID        bson.ObjectId   `json:"id" bson:"_id"`
  Name      string          `json:"name"`
  Level     int             `json:"level"`
  UserIDs   []bson.ObjectId `json:"userIDs" bson:"userIDs"`
  Users     []User          `json:"-" bson:"-"` // doesn't get stored in the database
  Documents []Document      // `json:"-" bson:"-" Removed this so that mgo can unmarshal
                            // the documents correctly
}

Now I just have to figure out a way to not store the Documents field in the database when I save a Folder.