For folks who have deployed HBase on their own clusters, do you feel that it's sufficiently stable for production use? What types of troubles or issues have you run into?
I do see a bunch of companies listed as using HBase in production (http://wiki.apache.org/hadoop/Hbase/PoweredBy), but I'm curious as to whether a lot of maintenance, patching, and firedrills goes into keeping the HBase cluster up and running.
HBase can be used without Hadoop. Running HBase in standalone mode will use the local file system. Hadoop is just a distributed file system with redundancy and the ability to scale to very large sizes.
HBase relies completely on Zookeeper. HBase provides you the option to use its built-in Zookeeper which will get started whenever you start HBAse. But it is not good if you are working on a production cluster.
HBase divides the logical table into multiple data blocks, HRegion, and stores them in HRegionServer. HMaster is responsible for managing all HRegionServers. It does not store any data itself, but only stores the mappings (metadata) of data to HRegionServer.
The HBase Master coordinates the HBase Cluster and is responsible for administrative operations. A Region Server can serve one or more Regions. Each Region is assigned to a Region Server on startup and the master can decide to move a Region from one Region Server to another as the result of a load balance operation.
HBase is about to hit a major milestone with HBase-0.20. There's is an alpha and soon to be a RC. It has had very major performance improvements. StumbleUpon reportedly serve their site live out the trunk version of HBase, with no additional caching layer, as do others. So I'd say it's definitely ready for production use.
Ryan Rawson (of StumbleUpon) gave a nice talk on it at the nosql conference recently, which mostly is about how far it's come over the last 6 months. There are slides if you don't want to watch the whole thing. Apart from performance improvements the other major addition is it integrates with zookeeper now, so the master isn't a single point of failure anymore.
HBase used to fall over with small cell sizes with memory issues because of a limitation of the file format. This has been addressed too with a new custom file format, which also gave performance gains.
I've been experimenting with HBase for about a year now, I'm ready to trust 0.20 with a production service, I wasn't quite with older versions. I recommended at least a 4 or 5 node devcluster when experimenting.
I can't really comment on what it's like care-taking a production cluster, because we only just started with a production one. An aspect that helps is the mailing list is extremely active and irc is in constant use so there's a very strong community for helping out at least.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With