Need help conceptualizing in Redis/NoSQL

Question

I think I have a good grasp of all the commands to use Redis, but I'm having difficulty figuring out the best way to use it. I'm designing a customer notification system that will notify them via their preferred method (Email, SNMP, Syslog) when there's an alarm on any of their circuits.

So, I get a device name and a port. I need to associate that with a single customer, and then associate that customer to a delivery method. With a relational db, it would look probably look something like this:

Device name: Los_Angeles
Port: 11

SELECT Customer_ID, Customer_name from device_info where device_port = 'Los_Angeles:11'
SELECT Customer_protocol, SNMP_destination, Syslog_destination from CUSTOMER 
    where Customer_ID = <customer_id from above>

(Greatly simplified example).

I can see how to do this programatically with a hash of lists or hash of hashes. But I guess what I'm having trouble with in Redis is that those more complex data structures aren't available to me (as far as I know). So, how do I associate multiple pieces of information with a single key? I can think of a few ways I can do it, but they all seem to involve multiple steps, and I'd like some input from current Redis programmers about what the "best" way to do this.

Answer 1

You are correct that only simple data structures are available with Redis, and they cannot be composed by value (like you could do with a document oriented database such as CouchDB or MongoDB). However, it is possible to compose data structures by reference, and this is a very common pattern.

For instance, the items contained in a set can be the keys for other objects (lists, hash tables, other sets, etc ...). Let's try to apply this to your example.

To model a relationship between customers and device+port, you can use sets containing customer IDs. To store information about the customers, one hash table per customer is fine.

Here are the customers:

hmset c:1 name Smith protocol tcp snmp_dest 127.0.0.1 syslog_dest 127.0.0.2
hmset c:2 name Jackson protocol udp snmp_dest 127.0.0.1 syslog_dest 127.0.0.2
hmset c:3 name Davis protocol tcp snmp_dest 127.0.0.3 syslog_dest 127.0.0.4

The keys of these records are c:ID

Let's associate two of them to a device and port:

sadd d:Los_Angeles:11 2 3

The key of this set is d:device:port. The c: and d: prefixes are just a convention. One set per device/port should be created. A given customer could belong to several sets (and therefore associated to several device/port).

Now to find the customers with delivery methods attached to this device/port, we just have to retrieve the content of the set.

smembers d:Los_Angeles:11
1) "2"
2) "3"

then the corresponding customer information can be retrieved by pipelining a number of hgetall commands:

hgetall c:2
hgetall c:3
1) "name"
2) "Jackson"
3) "protocol"
4) "udp"
5) "snmp_dest"
6) "127.0.0.1"
7) "syslog_dest"
8) "127.0.0.2"
1) "name"
2) "Davis"
3) "protocol"
4) "tcp"
5) "snmp_dest"
6) "127.0.0.3"
7) "syslog_dest"
8) "127.0.0.4"

Don't be afraid of the number of commands. They are very fast, and most Redis clients have the capability to pipeline the queries so that only a minimum number of roundtrips are needed. By just using one smembers and several hgetall, the problem can be solved with just two roundtrips.

Now, it is possible to optimize a bit further, thanks to the ubiquitous SORT command. This is probably the most complex command in Redis, and it can be used to save a roundtrip here.

sort d:Los_Angeles:11 by nosort get c:*->name get c:*->protocol get c:*->snmp_dest get c:*->syslog_dest
1) "Jackson"
2) "udp"
3) "127.0.0.1"
4) "127.0.0.2"
5) "Davis"
6) "tcp"
7) "127.0.0.3"
8) "127.0.0.4"

In one command, it retrieves the content of a device/port set and fetches the corresponding customer information.

This example was trivial, but more generally, while you can represent complex data structures with Redis, it is not immediate. You need to carefully think about the model both in term of structure and of data access (i.e. at design time, stick to your data AND your use cases).