Insert or update many documents in MongoDB

Question

Is there a way to insert or update/replace multiple documents in MongoDB with a single query?

Assume the following collection:

[
    {_id: 1, text: "something"},
    {_id: 4, text: "baz"}
]

Now I would like to add multiple documents of which some might already be in the collection. If the documents are already in the collection, I would like to update/replace them. For example, I would like to insert the following documents:

[
    {_id:1, text: "something else"},
    {_id:2, text: "foo"},
    {_id:3, text: "bar"}
]

The query should insert the documents with _id 2 and 3. It should also update/replace the document with _id 1. After the process, the collection should look as follows:

[
    {_id:1, text: "something else"},
    {_id:2, text: "foo"},
    {_id:3, text: "bar"},
    {_id:4, text: "baz"}
]

One approach might be to use insertMany:

db.collection.insertMany(
   [ {...}, {...}, {...} ],
   {
      ordered: false,
   }
)

If duplicates occur, that query will emit a writeErrors containing an array of objects containing the indexes of the documents that failed to insert. I could go through them and update them instead.

But that process is cumbersome. Is there a way to insert or update/replace many documents in one query?

Answer 1

As said here, to do what you need you can put something like this in

script.js

(* warning: untested code)

use YOUR_DB
var bulk = db.collection.initializeUnorderedBulkOp();
bulk.find( { _id : 1 } ).upsert().update( { $set: { "text": "something else" } } );
bulk.find( { _id : 4 } ).upsert().update( { $set: { "text": "baz" } } );
bulk.find( { _id : 99 } ).upsert().update( { $set: { "text": "mrga" } } );
bulk.execute();

and run it with

mongo < script.js

I had to do it this way as anything I tried for updating/inserting more than 1000 documents didn't work because of the limit.

Write commands can accept no more than 1000 operations. The Bulk() operations in the mongo shell and comparable methods in the drivers do not have this limit.

source

Answer 2

You can also use bulkWrite api to update or insert multiple documents in a single query, here is an example

var ops = []

items.forEach(item => {
    ops.push(
        {
            updateOne: {
                filter: { _id: unique_id },
                update: {
                    $set: { fields_to_update_if_exists },
                    $setOnInsert: { fileds_to_insert_if_does_not_exist }
                },
                upsert: true
            }
        }
    )
})

db.collections('collection_name').bulkWrite(ops, { ordered: false });

Answer 3

Considering data item as follows:

interface Item {
    id: string;         // unique key across collection
    someValue: string;
}

If you have items under the limit 1000, you can make bulk write operation like this:

public async insertOrUpdateBulk(items: Item[]) {
        try {
            const bulkOperation = this._itemCollection.initializeUnorderedBulkOp();
            for (let itemIndex = 0; itemIndex < items.length; itemIndex++) {
                const item = items[itemIndex];
                bulkOperation.find({ id: item.id }).upsert().update(item);
            }
            await bulkOperation.execute();
            return true;
        } catch (err) {
            console.log(err);
            return false;
        }
    }

If you have items that exceed the limit 1000, you can make simultaneous promises:

public async insertOrUpdate(items: Item[]) {
        try {
            const promises: Array<Promise<UpdateWriteOpResult>> = [];
            for (let itemIndex = 0; itemIndex < items.length; itemIndex++) {
                const item = items[itemIndex];
                const updatePromise = this.itemCollection.updateOne({ id: item.id }, item, { upsert: true });
                promises.push(updatePromise);
            }
            await Promise.all(promises);
            console.log('done...');
            return true;
        } catch (err) {
            console.log(err);
            return false;
        }
    }