pymongo DuplicateKeyError - during upsert

Question

I have noticed a strange behaviour of the pymongo library. When I run the following code from my unit test, everything is ok (the 2 documents are stored in their collection and the _id is set).

property_db_item = {#"_id": str(bson.ObjectId()),"url": item['url'],
                            "price": item['price'],"rooms": item['rooms'],
                            "area": item['area'],"flor": item['flor'],
                            "street": item['street'],"city": item['city'],
                            "region": item['region'],"country": item['country'],
                            "photoUrls": item['photoUrls'],"updatedAt": str(date)}

        db.properties.update({"url": item['url']}, property_db_item, upsert=True)
        db.property_trends.update(
            {"url": item['url'], "date": date.strftime("%Y-%m")},
            {
                "$setOnInsert": {"url": item['url'], #"_id": str(bson.ObjectId())],
                                 "date": date.strftime("%Y-%m")},
                "$addToSet": {"prices": {date.strftime("%Y-%m-%d"): item['price']}}

            }, upsert=True)`

but when I execute it from the terminal (scrapy crawl myParser), and get this error:

DuplicateKeyError(details["err"]) pymongo.errors.DuplicateKeyError: E11000 duplicate key error index: local.properties.$id dup key: { : null }

Answer 1

This is an existing bug. Upserts to local database fail with E11000 duplicate key error index: local.properties.$_id_ dup key: { : null } exception.

Here's an example:

> use local
switched to db local
> db.properties.update({'url': 'test1'}, {'url2': 'test2'}, upsert=true)
> db.properties.update({'url': 'test1'}, {'url2': 'test2'}, upsert=true)
E11000 duplicate key error index: local.properties.$_id_  dup key: { : null }
> use local2
switched to db local2
> db.properties.update({'url': 'test1'}, {'url2': 'test2'}, upsert=true)
> db.properties.update({'url': 'test1'}, {'url2': 'test2'}, upsert=true)