HBase schema/key for real-time analytics solution

Question

We are looking at using HBase for real-time analytics.

Prior to HBase, we will be running a Hadoop Map Reduce job over our log files and aggregating the data, and storing the fine-grained aggregate results in HBase to enable real-time analytics and queries on the aggregated data. So the HBase tables will have pre-aggregated data (by date).

My question is: how to best design the schema and primary key design for the HBase database to enable fast but flexible queries.

For example, assume that we store the following lines in a database:

timestamp, client_ip, url, referrer, useragent

and say our map-reduce job produces three different output fields, each of which we want to store in a separate "table" (HBase column family):

• date, operating_system, browser
• date, url, referrer
• date, url, country

(our map-reduce job obtains the operating_system, browser and country fields from the user agent and client_ip data.)

My question is: how can we structure the HBase schema to allow fast, near-realtime and flexible lookups for any of these fields, or a combination? For instance, the user must be able to specify:

• operating_system by date ("How many iPad users in this date range?")
• url by country and date ("How many users to this url from this country for the last month?")

and basically any other custom query?

Should we use keys like this:

• date_os_browser
• date_url_referrer
• date_url_country

and if so, can we fulfill the sort of queries specified above?

Answer 1

You've got the gist of it, yes. Both of your example queries filter by date, and that's a natural "primary" dimension in this domain (event reporting).

A common note you'll get about starting your keys with a date is that it will cause "hot spotting" problems; the essence of that problem is, date ranges that are contiguous in time will also be contiguous servers, and so if you're always inserting and querying data that happened "now" (or "recently"), one server will get all the load while the others sit idle. This doesn't sound like it'd be a huge concern on insert, since you'll be batch loading exclusively, but it might be a problem on read; if all of your queries go to one of your 20 servers, you'll effectively be at 5% capacity.

OpenTSDB gets around this by prepending a 3-byte "metric id" before the date, and that works well to spray updates across the whole cluster. If you have something that's similar, and you know you always (or usually) include a filter for it in most queries, you could use that. Or you could prepend a hash of some higher order part of the date (like "month") and then at least your reads would be a little more spread out.