Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MongoDB insertMany and skip duplicates

I'm trying to insertMany() items into my Mongo database but I would like to skip duplicate IDs.
I'm using Node.js and mongodb.

I have some data:

const myExampleData = [
   {_id:'someId1', name:'I am example'},
   {_id:'someId2', name:'I am second example'}
];

and I would like to insert them like this:

dbo.collection(collectionName).insertMany(myExampleData).catch(err=>{
    console.error(err);
});

Lets suppose that someId1 already exists. I don't want to override it. I just want to skip it. In current situation it doesn't insert someId2. It stops as soon as it throws duplicate exception.

Is there a way how to insertMany and skip duplicates?


Possible question duplicate.

I've found thread MongoDB insert without duplicates where is suggested to use update() with upsert instead of insert() which could be fine for one item. But how about many items? As far as I know updateMany() would update all filtered rows with the same value but I would like to insert different values.

like image 436
Jax-p Avatar asked Apr 28 '20 12:04

Jax-p


1 Answers

Checking in your statement :

Lets suppose that someId1 already exists. I don't want to override it. I just want to skip it.

So you just wanted to skip duplicate docs as your intention is not update duplicate docs with latest data - So there is no need to use .update(), You can still do this using .insertMany() by passing in a flag ordered in options to query :

Ordered : Optional. A boolean specifying whether the mongod instance should perform an ordered or unordered insert. Defaults to true.

db.collection.insertMany(
   [ <document 1> , <document 2>, ... ],
   {
     ordered: <boolean>
   }
)

Your code :

dbo.collection(collectionName).insertMany(myExampleData, {ordered : false }).catch(err=>{
    console.error(err);
})

As if you're checking against _id which will have default unique index & any incoming duplicates will actually throw an error, With ordered : false we're making this insertion operation un-ordered by that we're skipping all the incoming duplicates & proceeding further with out actually throwing any errors.

like image 73
whoami - fakeFaceTrueSoul Avatar answered Oct 12 '22 11:10

whoami - fakeFaceTrueSoul