MongoDB to Use Sharding with $lookup Aggregation Operator

Question

$lookup is new in MongoDB 3.2. It performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing.

To use $lookup, the from collection cannot be sharded.

On the other hand, sharding is a useful horizontal scaling approach.

What's the best practise to use them together?

Answer 1

As the docs you quote indicate, you can't use $lookup on a sharded collection. So the best practice workaround is to perform the lookup yourself in a separate query.

1. Perform your aggregate query.
2. Pull the "localField" values from your query results into an array, possibly using Array#map.
3. Perform a find query against the "from" collection, using a query like {foreignField: {$in: localFieldArray}}
4. Merge your results into whatever format you need.

Don't let the $lookup limitation stop you from sharding collections that require it for scalability, just perform the lookup function yourself.