Mongo - How can I aggregate, filter, and include an array of data from the matching documents?

Question

I have a mongo-backed contact database going and I'm trying to find duplicate entries in a bunch of different ways.

For example, if 2 contacts have the same phone number they are flagged as a possible duplicate, ditto for email, etc.

I'm using MongoDB 2.4.2 on Debian with pyMongo and MongoEngine.

The closest I have so far is finding and counting records that contain the same phone number:

dbh.person_document.aggregate([
    {'$unwind': '$phones'},
    {'$group': {'_id': '$phones', 'count': {'$sum': 1}}},
    {'$sort': SON([('count', -1), ('_id', -1)])}
])

# Results in 
{u'ok': 1.0,
 u'result': [{u'_id': {u'number': u'404-231-4444', u'showroom_id': 5}, u'count': 5},
             {u'_id': {u'number': u'205-265-6666', u'showroom_id': 5}, u'count': 5},
             {u'_id': {u'number': u'213-785-7777', u'showroom_id': 5}, u'count': 4},
             {u'_id': {u'number': u'334-821-9999', u'showroom_id': 5}, u'count': 3}
]}

So I can get the numbers that are duplicates, but I can't for the life of me figure out how to return an array of the Documents that actually contained these items!

I wanna see this kind of return data for each number:

# The ObjectIDs of the documents that contained the duplicate phone numbers
{u'_id': {u'number': u'404-231-4444', u'showroom_id': 5}, 
  u'ids': [ObjectId('51c67e322b2192121ec4d8f2'), ObjectId('51c67e312b2192121ec4d8f0')], 
  u'count': 2},

Any help is greatly appreciated!

Answer 1

Ah, blessed be.

Found the solution almost word for word at MongoDB - Use aggregation framework or mapreduce for matching array of strings within documents (profile matching) .

Final result, adding some extra to include the name:

dbh.person_document.aggregate([
    {'$unwind': '$phones'},
    {'$group': {
        '_id': '$phones',
        'matchedDocuments': {
            '$push':{
                'id': '$_id',
                'name': '$full_name'
                }},
        'num': { '$sum': 1}
    }},
    {'$match':{'num': {'$gt': 1}}}
])