Mongodb Aggregation count array/set size

Question

Here's my problem:

Model:

{ application: "abc", date: Time.now, status: "1" user_id: [ id1, id2, id4] }

{ application: "abc", date: Time.yesterday, status: "1", user_id: [ id1, id3, id5] }

{ application: "abc", date: Time.yesterday-1, status: "1", user_id: [ id1, id3, id5] }

I need to count the unique number of user_ids in a period of time.

Expected result:

{ application: "abc", status: "1", unique_id_count: 5 }

I'm currently using the aggregation framework and counting the ids outside mongodb.

{ $match: { application: "abc" } }, { $unwind: "$users" }, { $group: { _id: { status: "$status"}, users: { $addToSet: "$users" } } }

My arrays of users ids are very large, so I have to iterate the dates or I'll get the maximum document limit (16mb).

I could also $group by

{ year: { $year: "$date" }, month: { $month: "$date" }, day: { $dayOfMonth: "$date" }

but I also get the document size limitation.

Is it possible to count the set size in mongodb?

thanks

Answer 1

The following will return number of uniqueUsers per application. This will apply an group operation to a result of a group operation by using pipeline feature of mongodb.

{ $match: { application: "abc" } }, 
{ $unwind: "$users" }, 
{ $group: { _id: "$status", users: { $addToSet: "$users" } } }, 
{ $unwind:"$users" }, 
{ $group : {_id : "$_id", count : {$sum : 1} } }

Hopefully this will be done in an easier way in the following releases of mongo by a command which gives the size of an array under a projection. {$project: {id: "$_id", count: {$size: "$uniqueUsers"}}} https://jira.mongodb.org/browse/SERVER-4899

Cheers

Answer 2

Sorry I'm a little late to the party. Simply grouping on the 'user_id' and counting the result with a trivial group works just fine and doesn't run into doc size limits.

[
    {$match: {application: 'abc', date: {$gte: startDate, $lte: endDate}}},
    {$unwind: '$user_id'},
    {$group: {_id: '$user_id'}},
    {$group: {_id: 'singleton', count: {$sum: 1}}}
];