MongoDB: How to design twitter-style followers/following relation model in MongoDB?

Question

I'm developing an android mobile app which potentially have a lot of users (let's say about 1 million). These user can follow other users (like Twitter). The app syncs the user data via a remote REST backend. The user data itself is persisted in a document oriented database (in my case it's a MongoDB).

Currently I ask myself about the best way to design the user model including its follower and following relations. First thought was to embed the relations within the user document.

Example user document:

{
"_id":"50fd6bb530043e3c569af288", 
"name":"Marsha Garcia", 
"follower"["50fd6bb530043e3c569af287","50fd6bb530043e3c569af289","50fd6bb530043e3c569af28c"],
"following":["70fd6bb530043e3c569af289","10fd6bb530043e3c569af222","89fd6bb530043e3c569af45o"]
}

The positive thing is that the follow/following relations are already joined with the user. However let's say a user follows about 100.000 or more other users. Then the document size will become very large. If I load this user object via REST service in my mobile app it could take a while. Furthermore in worst case the user document could exceed the MongoDb 16MB document limit.

Therefore my second thought was to model the follower and following relations in a more classic way: An extra document containing the following relations of each user.

Example 'user relation' document:

{
    "_id": 50fe65828de290c0a8a8ea2d"
    "uid": "50fd6bb530043e3c569af288",
    "rel_uid": "50fe65828de290c0a8a8e9a6",
    "type": "FOLLOWING"
}

The positive thing is that the size of each user document will remain constant. The downside is that with lots of users and following relations I could easily get millions of entries in my MongoDB 'user relations' collection. Of course I'm going to set an index on the fields but I'm not quite sure whether this solution will scale very well regarding the use case of an app user asking for his/hers current followers.

I would appreciate any thoughts, experiences about my modeling problem. Perhaps anyone even has a better solution approach.

Thx a lot in advance.

Answer 1

1. collection users:
- userid
- username
- userpass
- other user specific info user


2. collection following:
- userid
- [array of followingid]


3. collection followed:
- userid
- [array of followedid]
4. messages_relation collection:

- userid
- messageid
- time

5. messages_text:
- messageid
- text

Answer 2

I'd start by reading this documentation on storing comments in a CMS if you haven't already. While it's for comments, the same general problem exists -- where you can't store all comments in a single document (in your case, followers/following).

Either the Hybrid approach (which uses fewer docs and stores some number of relations inside of a single doc) or the approach you're describing should work well.

I'd also suggest building a simple POC to test the performance of retrievals and such. It might make sense to cache some results, or precompile them. Usually, it's OK in systems like this if everything isn't instantly consistent for all users (like having the follower count be correct right away).

There likely isn't a perfect solution, and may require a few solutions for optimal performance (like how a user and followers are handled may change as the number of followers increases for example).