Is Zookeeper appropriate for object caching?

Question

Developing a cloud system with an eye on scalability meaning that the system is composed into REST-based services along functional boundaries (usermgmt, ordermgmt, customermgt etc) each with its own underlying database and depending on load, its presumable that we might spins up multiple instances of say the ordermgmt service. When the ordermgmt service processes a request to add an order (on behalf of a "customer"), it will make a REST call to the customermgmt service to validate the customer etc...

Since a customer entity would not change very often, I'm wondering if something like ZooKeeper would be appropriate to cache an instance of a particular customer that multiple instances of a customermgmt service might interrogate before hitting the database. I've looked at the various lists of Zookeeper uses but don't see anyone using it for object caching. Seems like the recommended znode size of bytes to approx 1K making it not appropriate for storing an dehydrated object. Also there's no support for GC or LRU out of the box so I would need to add this as well.

If not Zookeeper, any more appropriate suggestions? We're using Hibernate as the ORM, but we don't have a lot of experience with it and while it support 1st and 2nd level caches, I'm not sure if there they work in a distributed/replicated manner across multiple service instances.

Thanks Scott

Answer 1

Zookeeper is not a great fit for an object cache.

Zookeeper keeps the entire database in memory in the java heap. Once the java heap gets over a Gigabyte or so, you will start running into problems with gc pauses. This is especially troublesome with zookeeper because the zookeeper nodes are sending hearbeats to each other constantly, and if enough heartbeats are missed while a node is busy gc'ing, a leader election is triggered, bringing the cluster down momemtarily.

The other problem with using zookeeper as a cache is that all nodes in a zookeeper cluster will have the same data, which you typically don't need for a cache.

With these limitations, your 3 servers, each with 8 gigs of ram, manage to serve a total working set of ~ 1 Gig. Better to use memcache, or one of the other caching system Sebastien lists.

Answer 2

Actually you can set many different technologies, distributed or not, as the L2 cache of Hibernate.

• EhCache
• Memcached
• JCache
• Hazelcast
• Infinispan
• Terracotta
• Gigaspaces XAP
• Gemfire
• Coherence

The last ones, called datagrids, are usually not free and i don't think you need a whole datagrid just for a l2 cache.

I never used it but i don't think Zookeeper was made to be used as a distributed cache.