insertMany always returns duplicate _id error

Question

I have a Mongo collection that I'm trying to insert multiple documents into, as below:

db.collection('properties').insertMany(docs)
  .catch(err => console.log(err))
  .then((err, result) => {
    console.log(err);
    console.log(docs);
    console.log(result);
    //if (err) console.log(err);
    //else if (callback) callback();
  });

This always returns the following error:

{ [MongoError: insertDocument :: caused by :: 11000 
   E11000 duplicate key error index: properties.properties.$_id_  
    dup key: { : ObjectId('591bbecdf9d86c59eea1047c') }]

None of the objects within the array initially have _id properties. However, the console.log(docs) shows afterwards:

 { url: '/property/z37717098',
    thumbnailUrl: 'https://li...2e175fca56f08ceb6ffab5_354_255.jpg',
    lat: '50.81647',
    lng: '-1.085111',
    dateAdded: '30/07/2015',
    images: 
     [ 'https://li.zoocdn...b2e175fca56f08ceb6ffab5_645_430.jpg',
       'https://li.zoocdn...c2e77d50e653300e5d21358d4f9825_645_430.jpg',
       'https://li.zoocdn...523163e4684226420bb4c167d90666_645_430.jpg',
       'https://li.zoocdn...e008165a61d9154a7a59c881_645_430.jpg',
       'https://li.zoocd...218736df8f964745602744f7c_645_430.jpg',
       'https://li.zoo...ad5a98ed2ffe78322c2_645_430.jpg',
       'https://li.zooc...efe4a54042ff76690_645_430.jpg',
       'https://li.zoocd...6813d439a1740f42e_645_430.jpg',
       'https://li.zooc...c2b38417154aeba6c28cbd_645_430.jpg' ],
    _id: 591bbecdf9d86c59eea1047c },

Sure enough, all the object in the array now have an _id property with a duplicate value for each object.

EDIT: I should also mention that the first object in the array is inserted to the collection, and has the same value for its _id property as the duplicate error message displays as a conflict. This implies that to me that the insertMany function is giving the objects the same ObjectId.

What's going on? Why is insertMany generating a whole bunch of duplicate ObjectId's when that will obviously cause the insert to fail?

Answer 1

I was having the same problem with insertMany(). I couldn't identify the root cause. In my case I was looping over arrays with objects (documents), and inserting them in bulk. Anywhere from a few hundred to several thousand records at a time.

In my reading, I'm guessing it has something to do with the driver—as ultimately it's what's adding the _id field for you before insert. Though, I didn't spend to much time digging into this.

Again, in my experience I always ended up with E11000 duplicate key errors. So, as a workaround I ended up programmatically creating my own _id field. It's entirely up to you how you'd like to implement it.

In my case I just imported the uuid library:

npm install uuid

Then in I looped over all my documents, and appended the uuid.

documents.forEach(doc => doc._id = uuidv1())

Hope that helps!