Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Performance implications of db.collection.insert vs db.collection.insertOne & db.collection.insertMany

I am using mongodb driver for NodeJs, there are 3 methods in it:

1) db.collection.insert

2) db.collection.insertOne

3) db.collection.insertMany

I find that db.collection.insert does the job of both insertOne as well as insertMany.

I also find the same methods for delete & Updates.

Are there any performance impact in calling the db.collection.insert method vis-a-vis the db.collection.insertOne & db.collection.insertMany methods ?

It is safe to assume that I am working in a collection which will have million records at some point in time.

like image 969
Abdul Rehman Sayed Avatar asked May 30 '16 10:05

Abdul Rehman Sayed


People also ask

Is insertMany faster than insertOne?

If you have multiple documents to insert, insertMany is better, faster & recommended.

What is difference between insert insertOne and insertMany?

With insertOne you can insert one document into the collection. insertMany accepts an array of documents and these are inserted. The insert (the method from older versions) takes a single document by default, and there is an option to insert multiple documents supplied as an array.

What does insertMany return?

The insertMany() method returns a document that contains: The acknowledged key sets to true if operation executed with a write concern or false if the write concern was disabled. An array of _id values of successfully inserted documents.


4 Answers

There are some minor differences:

  • db.collection.explain() does not work for insertOne() or insertMany(), but for insert()
  • insertMany() and insert() can be ordered (default) or unordered. In latter case mongo may perform reordering for performance reasons.
  • insert() returns a WriteResult document instead of a status document (containing the new objectIds)

So it depends on your access pattern which to choose. In general, for performance, if you have to add multiple documents, try to make 1 call only (insert() or insertMany()) and have it unordered (if possible from application point of view). If it's a question on whether you should use (insert() or insertMany()) it depends on if you need explain() or the resulting objectIds, but there are no differencese regarding performance (if you type: db.yourCollection.insert or db.yourCollection.insertMany (without ()) you'll see that both perform a bulk.insert(obj))

TL;DR:

  • insert 1 document: insertOne() or insert()
  • insert n documents: insertMany() or insert()
  • insert documents unordered: insertMany() or insert() + ordered:false
  • you need status of operation with ObjectIds: insertOne() or insertMany()
  • you need WriteResult: insert()
  • you need explain(): insert()
like image 109
Gerald Mücke Avatar answered Sep 30 '22 13:09

Gerald Mücke


When performing operation let say on 30 documents it is more efficient to use insert/update with many option as:

  1. there is one call to server
  2. server engine processes work without waiting for new data

so reducing roundtrips, network overhead and allows db engine to process all documents at once give listed advantages over looping via all docs on client side calling insert/update one by one

like image 36
profesor79 Avatar answered Sep 30 '22 14:09

profesor79


Performance Test

  • 3.7k x 50000 data
  • Tried to consider same condition
  • Meteor driver wraped up Node driver for Fiber to write a code

Node Driver Test

  • Node driver insertMany with async + Promise sync: 10~11 sec
  • Node driver insertMany with async: 13~15 sec
  • Node driver insert with async: 20~23 sec

Meteor Driver Test

  • Meteor driver insert with async: 24~27 sec
  • Meteor driver insert with sync: 43-49 sec

Conclusion

  • insertMany is much faster than insert
  • Promise + await is faster than async
    • Can't explain. Any Idea?
  • Meteor sync function is slow for bulk insert
like image 40
kakadais Avatar answered Sep 30 '22 12:09

kakadais


It was a 10-fold difference performance-wise bw insert_one() and insert_many() in my tests.

I am using Python 3.9 and MongoDB 5.0, I am inserting each line of a CSV file of ~5M lines/~500MB as one document into a MongoDb collection. It resulted in a 4.8GB collection, each document about 1KB as reported by MongoDB Compass.

It took:

  • 3048s with insert_one()
  • 301s with insert_many() with batches of 1,000 rows
  • 291s with insert_many() with batches of 5,000 rows
  • 273s with insert_many() with batches of 10,000 rows
like image 21
lcfd Avatar answered Sep 30 '22 12:09

lcfd