Mongodb bulk insert limit in Python

Question

Is there a limit to the number of documents one can bulk insert with PyMongo? And I don't mean the 16mb limit of document size for MongoDB, but the actual size of the list of documents I wish to insert in bulk through Python.

Answer 1

There is no limit on the number of documents for bulk insert via pymongo. According to the docs, you can provide an iterable to the collection.insert, and it will

insert each document in the iterable, sending only a single command to the server

Key point here is that pymongo will try to do your insert by sending one single message to the mongodb server.

Mongodb itself has a message size limit (maxMessageSizeBytes), that is equal to 48000000 bytes (maxBsonObjectSize * 3).

So, pymongo client driver should be responsible for splitting your large message into smaller messages to fit into mongodb max size limit. But, actually it's not yet implemented. See:

• https://jira.mongodb.org/browse/PYTHON-414
• https://jira.mongodb.org/browse/PYTHON-419

For now, you have to handle this situation by yourself.

Hope that helps.