MongoDB $slice (embedded array paging)

Question

I have this schema:

article: {
    subject,
    comments: []
}

if I have 8 comments, and query

 article.find({}, {
     comments: {
         $slice: [ -10, 5 ]
     }
 });

And I get comments from index 0 to index 4,
but I only want comments from index 0 to index 2 to be returned because of paging.
(page 1 $slice[ -5, 5 ] from index 3 to index 7, page 2 $slice[ -10, 5 ] from index 0 to index 2)

now I have to pass another parameter "lastId" to compare each comments and remove that "_id" < "lastId", but I think it is a little hacky.

Any one has a good solution for this?

Answer 1

So I'm going to say that you should switch your schema to leave comments as separate documents as this is an unbound array and it will make your queries more efficient. I'll explain it.

When you add embedded documents to an array that is not a fixed size mongoDB will potentially need to move the document around as it grows, changing the padding factor and causing fragmentation (the padding factor is a guess from mongodb's side off how big your document will grow, it preallocates more space for that case).

You are also limited to 16MB pr document so imagine if you get a crazy popular thread or you decide to extends comments with other metadata it's feasible that you'll break that barrier. Retrieving a big document is also expensive and time consuming.

In general embedded documents are great if they are not unbound arrays. So keeping a list of the top 10 comments will work great but keeping 1000+ comments is bad.

There are some good presentation under

http://www.10gen.com/presentations/mongodb-berlin/2012/10-key-performance-indicators http://www.10gen.com/presentations/mongosv-2011/schema-design-by-example

I think there is more work coming soon on schema design that will more helpful in the long term. I think it's the hardest bit to be honest. I know, it took me awhile to wrap my head around the differences from relational models.