Inserting multiple documents into mongodb using one call in Meteor

Question

In the mongo shell, it is possible to insert an array of documents with one call. In a Meteor project, I have tried using

MyCollection = new Mongo.Collection("my_collection")
documentArray = [{"one": 1}, {"two": 2}]
MyCollection.insert(documentArray)

However, when I check my_collection from the mongo shell, it shows that only one document has been inserted, and that document contains the entire array as if it had been a map:

db.my_collection.find({})
{ "_id" : "KPsbjZt5ALZam4MTd", "0" : { "one" : 1 }, "1" : { "two" : 2} }

Is there a Meteor call that I can use to add a series of documents all at once, or must use a technique such as the one described here?

I imagine that inserting multiple documents in a single call would optimize performance on the client side, where the new documents would become available all at once.

Answer 1

You could use the bulk API to do the bulk insert on the server side. Manipulate the array using the forEach() method and within the loop insert the document using bulk insert operations which are simply abstractions on top of the server to make it easy to build bulk operations.

Note, for older MongoDB servers than 2.6 the API will downconvert the operations. However it's not possible to downconvert 100% so there might be some edge cases where it cannot correctly report the right numbers.

You can get raw access to the collection and database objects in the npm MongoDB driver through rawCollection and rawDatabase methods on Mongo.Collection

MyCollection = new Mongo.Collection("my_collection");

if (Meteor.isServer) {
    Meteor.startup(function () {
        Meteor.methods({
            insertData: function() {
                var bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp(),
                    counter = 0,
                    documentArray = [{"one": 1}, {"two": 2}];

                documentArray.forEach(function(data) {
                    bulkOp.insert(data);

                    counter++;
                    // Send to server in batch of 1000 insert operations
                    if (counter % 1000 == 0) {
                        // Execute per 1000 operations and re-initialize every 1000 update statements
                        bulkOp.execute(function(e, rresult) {
                            // do something with result
                        });
                        bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp();
                    }
                }); 

                // Clean up queues
                if (counter % 1000 != 0){
                    bulkOp.execute(function(e, result) {
                        // do something with result
                    });
                }
            }
        }); 
    });
}

Answer 2

I'm currently using the mikowals:batch-insert package.

Your code would work then with one small change:

MyCollection = new Mongo.Collection("my_collection");
documentArray = [{"one": 1}, {"two": 2}];
MyCollection.batchInsert(documentArray);

The one drawback of this I've noticed is that it doesn't honor simple-schema.