How to handle DuplicateKeyError in MongoDB (pyMongo)?

Question

Could anybody please tell me how to handle DuplicateKeyError in MongoDB?

I am writing a python script, where I move several docs from two different collections into a third one. There is a small overlap between the two collections due to having a few identical documents (with identical ObjectId). This results in the following:

DuplicateKeyError: E11000 duplicate key error collection: admin.collection_test index: id dup key: { : ObjectId('593a920b529e170d4b8fbf72') }

In order to get rid of the error I use:

try:
    do something
except pymongo.errors.DuplicateKeyError:
    pass

I expect by using of the "try-except" to move all non-crossing documents to the third collection, but instead the script just peacefully stops running once a first overlap (an already existing document in the collection) appears. Would appreciate any help a lot!

Answer 1

If you're iterating over the documents, try using continue instead of pass.

for doc in documents:
    try:
        # insert into new collection
    except pymongo.errors.DuplicateKeyError:
        # skip document because it already exists in new collection
        continue

Answer 2

for doc in documents:
    client.update_one({'_id': doc['_id']}, doc, upsert=True)

You can use update_one with upsert=True. This updates doc with new doc if doc exists already otherwise it creates new doc.