Does it make sense to use both redis and mongodb?

Question

We have a lot of data, decided to use mongodb and it works great.

We started using redis to track the active users in our real-time app. We also started doing some pub/sub channel stuff with redis.

Our next move might be to use mongodb for dormant data and redis for active data. An example of this would be, all of our users are stored in mongodb but when they are logged in we will move a copy of that data to redis for fast access. We also store things like their game activity in redis and use the data accordingly. When the user logs out we will save anything needed in mongo where it will live until its needed again and loaded into redis.

One thing we have been looking into is preservation of redis on crash. User activity on the system is meaningful data that we wouldn't want to lose on crash, and if we are only logging data after the fact, should we save a back up of important data in mongo after every event? Then on crash redis can restore from mongo?

Is there are better way to go about the things we are trying to achieve?

Thanks!

Answer 1

OK, so there are several angles from which to attack this question. The first thing to point out is that redis does have user-configurable persistence.

User activity on the system is meaningful data that we wouldn't want to lose on crash, and if we are only logging data after the fact, should we save a back up of important data in mongo after every event?

To be fair, the default setup with MongoDB is to flush to disk every 60 seconds. So you still have a 60 second window of data loss.

1. You can use journaling and drop that window to 100ms, but that will tax the IO more heavily.
2. You can also configure your writers to wait on that journal to flush (WriteConcern: fsync), but that's going to slow down writes significantly.

Is there are better way to go about the things we are trying to achieve?

Really depends on what you're trying to achieve.

• What type of data loss can you handle?
• Redis has replication, are you using that? Does that solve most of your data loss worries?
• You say you're using PubSub features, how many nodes does this cover? Is your data adequately replicated just as a result of this?

Either way, it's a somewhat complex problem. MongoDB may kind of solve your problems, but replication may solve those problems just as well. Depends on your comfort level.