mongodb limit in the embedded document

Question

I need to create a message system, where a person can have a conversation with many users. For example I start to speak with user2, user3 and user4, so anyone of them can see the whole conversation, and if the conversation is not private at any point of time any of participants can add any other person to the conversation.

Here is my idea how to do this. I am using Mongo and my idea is to use dialog as an instance instead of message.

The schema is listed as follows:

{
_id : ...., // dialog Id
'private' : 0 // is the conversation private
'participants' : [1, 3, 5, 6], //people who are in the conversation
'msgs' :[
  {
   'mid' : ...// id of a message
   'pid': 1, // person who wrote a message
   'msg' : 'tafasd' //message
  },
  ....
  {
   'mid' : ...// id of a message
   'pid': 1, // person who wrote a message
   'msg' : 'tafasd' //message
  }
]
}

I can see some pros for this approach - in a big database it will be easy to find messages for some particular conversation. - it will be easy to add people to the conversation.

but here is a problem, for which I can't find a solution: the conversation is becoming too long (take skype as an example) and they are not showing you all the conversation, they are showing you a part and afterwards they are showing you additional messages. In other situations skip, limit solves the case, but how can I do this here?

If this is impossible what suggestions do you have?

Answer 1

The MongoDB docs explain how to select a subrange of an array element.

db.dialogs.find({"_id": [dialogId]}, {msgs:{$slice: 5}}) // first 5 comments
db.dialogs.find({"_id": [dialogId]}, {msgs:{$slice: -5}}) // last 5 comments
db.dialogs.find({"_id": [dialogId]}, {msgs:{$slice: [20, 10]}}) // skip 20, limit 10
db.dialogs.find({"_id": [dialogId]}, {msgs:{$slice: [-20, 10]}}) // 20 from end, limit 10

You can use this technique to only select the messages that are relevant to your UI. However, I'm not sure that this is a good schema design. You may want to consider separating out "visible" messages from "archived" messages. It might make the querying a bit easier/faster.

Answer 2

There are caveats if your conversation will have many many messages:

1. You will notice significant performance reduction on slicing messages arrays as mongodb will do load all of them and will slice the list before return to driver only.
2. There is document size limit (16MB for now) that could be possibly reached by this approach.

My suggestions is:

1. Use two collections: one for conversations and the other for messages.
2. Use dbref in messages to conversation (index this field with the message timestamp to be able to select older ranges on user request).
3. Additional use separate capped collection for every conversation. It will be easy to find it by name if you build it like "conversation_"

Result:

• You will have to write all messages twice. But into separate collections which is normal.
• When you want to show your conversation you will need just to select all the data from one collection in natural sort order which is very fast.
• Your capped collections will automatically store last messages and delete old.
• You may show older messages on the user request by querying main messages collection.