Redis, Mongo or Hazelcast?

Question

We have a JAVA web app which uses postgres(single database with a slave) for storing all the important data.

We are now moving from a single server setup to mutiple servers due to which i need to make some changes to address the new requirements.

1) Non sticky session ids for load balancing and partition tolerance.

2) Cache of frequently read data accessible from all web servers (In Memory/Memcache alternative).

3) Queues(Email, SMS, Tasks to executed over the cluster). Typically all of them have to be executed over an xml api or screen scraping.
Avoiding duplicate processing of tasks is important but sometimes it can happen :-)

4) Persistent storage of API requests and responses(lots of XML, lots of rows but small number of columns). (probably archive by removing old requests and responses to keep the data set small).

5) Logging to a common place. The table will keep on growing. Also i would need a tool to access logs of production without stopping them. Some sort of search should be possible based on time and or search string.

I want a single solution to address all these requirements and looking at redis, mongo and hazelcast(in order of my personal preference) as possible alternatives.

Other important considerations: 1) Less intrusion into our code. 2) Easy backup/replication strategies. At least Master Slave. 3) Manageability, Community and tried and tested(running in production).

Which will be able to perform all or most out of this features and requirements?

EDIT - What i did

1. Redis backed session manager for tomact.
2. Redis for caching
3. Jesque (Respue's java version) backed by redis.
4. Postgres
5. SLF4J backed by Log4j2

Answer 1

I can address some of these from MongoDB's perspective.

The first thing I noticed is that you are moving from a single server setup to a multiple server setup. MongoDB makes it incredibly easy to setup replication and sharding. In turn, replication and sharding, along with some of Mongo's other features, can help you achieve a lot what you are setting out to do.

First, take a look at the documentation for a bit to get a feel for it:

Replica Sets and Sharding

Some other thoughts based on your requirements:

• Compared to other methods of scaling with different datastores mongo's method of horizontally scaling with commodity hardware is very simple to setup, scale and maintain. This means you can spend more time on building your app instead of being a DBA.
• You may also be able to skip a caching layer if you go with mongo. MongoDB uses memory mapped files which means that if your working set can be held in physical memory you basically have an in memory cache already.
• MongoDB is very good for logging. Users typically don't need safe writes for this sort of application so performance will be excellent if you stick with the default fire-and-forget model for writing.
• Whether this means that it will intrude in your code less is up for debate, however, compared to what an typical object relation mapper does, Mongo is much less intrusive to your data. It is able to store the data in it's natural usable state, the object!

Hope that helps, cheers.