Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to ignore duplicate documents when using insertMany in mongodb php library?

I am using mongo php library, and trying to insert some old data into mongodb. I used insertMany() method and pass a huge array of document, that may have duplicate documents on unique indexes.

Lets say I have a users collection and have these indexes:

[     {         "v" : 1,         "key" : {             "_id" : 1         },         "name" : "_id_",         "ns" : "test.users"     },     {         "v" : 1,         "unique" : true,         "key" : {             "email" : 1         },         "name" : "shop_id_1_title_1",         "ns" : "test.users"     } ] 

If there's a duplicate document, MongoDB\Driver\Exception\BulkWriteException would raise and stop the process. I want to find a way to ignore inserting duplicate documents (and also preventing the exception from raise) and continue inserting other documents.

I found in php.net documentation a flag called continueOnError that do the trick but it seems that it's not working with this library.

The example from php.net:

<?php  $con = new Mongo; $db = $con->demo;  $doc1 = array(         '_id' => new MongoId('4cb4ab6d7addf98506010001'),         'id' => 1,         'desc' => "ONE", ); $doc2 = array(         '_id' => new MongoId('4cb4ab6d7addf98506010002'),         'id' => 2,         'desc' => "TWO", ); $doc3 = array(         '_id' => new MongoId('4cb4ab6d7addf98506010002'), // same _id as above         'id' => 3,         'desc' => "THREE", ); $doc4 = array(         '_id' => new MongoId('4cb4ab6d7addf98506010004'),         'id' => 4,         'desc' => "FOUR", );  $c = $db->selectCollection('c'); $c->batchInsert(     array($doc1, $doc2, $doc3, $doc4),     array('continueOnError' => true) ); 

And the way I tried to use the flag with mongo php library:

<?php  $users = (new MongoDB\Client)->test->users  $collection->insertMany([     [         'username' => 'admin',         'email' => '[email protected]',         'name' => 'Admin User',     ],     [         'username' => 'test',         'email' => '[email protected]',         'name' => 'Test User',     ],     [         'username' => 'test 2',         'email' => '[email protected]',         'name' => 'Test User 2',     ], ], [     'continueOnError' => true    // This option is not working ]); 

The code above still raise the exception, and seems not to work. Is there other option flag or is there any way to do this?

like image 245
Behzadsh Avatar asked Nov 25 '16 19:11

Behzadsh


People also ask

How do I stop MongoDB from inserting duplicate records?

To insert records in MongoDB and avoid duplicates, use “unique:true”.

What is the function of insertMany () in MongoDB?

The insertMany() method inserts one or more documents in the collection. It takes array of documents to insert in the collection. By default, documents are inserted in the given order if you want to insert documents in unordered, then set the value of ordered to false.

How do I duplicate a document in MongoDB?

To clone a document, hover over the desired document and click the Clone button. When you click the Clone button, Compass opens the document insertion dialog with the same schema and values as the cloned document. You can edit any of these fields and values before you insert the new document.


1 Answers

Try to replace 'continueOnError' option by 'ordered' set to false, based on the documentation, when ordered option is set to false the insertMany will continue writing, even if a single write fails.

here is the docs link: insertMany

like image 94
YouneL Avatar answered Sep 21 '22 22:09

YouneL