MongoDB: $push Multiple Objects, and $pop Multiple Objects in Single Update

Question

Imagine a MongoDB document with an array of 100 objects. And we want to keep the array length fixed at 100. When a batch of new objects arrives (could be 1, 5, 10, etc.), we want to update the array with new objects, while removing an equal amount of old objects so the array length will stay fixed. I've opted to reading the array from MongoDB into my app, making some modifications, then using $set to update the array:

var newData = [
  { ... },
  { ... },
  { ... },
  { ... },
  { ... }
];
var oldData = Collection.findOne({exchange: 'The Exchange', market: 'The Market'}).data;
newData = newData.concat(oldData).slice(0, 100);
Collection.update(
  {
    exchange: 'The Exchange',
    market: 'The Market'
  },
  {
    $set: {
      data: newData
    }
  }

Is it possible in MongoDB to $push the new on to the front, while simultaneously using $pop to remove an equal amount of objects off the back?

Answer 1

Well the answer is both "yes" and "no" to put it in slightly confusing terms. What you cannot do is both $push and $pull operations on the same array in a singular update. This is not allowed for the "same path" of operations because neither $push or $pull is really determined to occur in any order within the current update syntax.

However in your specific context, that is not what you are asking. To do what you want, MongoDB supports the $slice modifier which can be used along with $each. This will effectively "limit" the total size of the array as new items are added to it.

Following your example:

Collection.update(
    { "exchange": "The Exchange", "market": "The Market" },
    { "$push": { "data": { "$each": newData, "$slice": 100 } } }
)

That effectively limits the size of the array to the first 100 members whilst adding the new items to it.

Take note though that this may not be supported in the client version of "minimongo" as implemented by meteor. What you can do though is execute this on the server, and expose the method via publish. There are plenty of examples to show you how to use publish.

Answer 2

Is it possible in MongoDB to $push the new on to the front, while simultaneously using $pop to remove an equal amount of objects off the back?

Yes, using $slice:

$push: {
    data: {
       $each: [ { ... }, { ... } ], // your batch
       $slice: -100 // maximum array size
    }
 }

For an example, see http://docs.mongodb.org/manual/tutorial/limit-number-of-elements-in-updated-array/