Using memcached as a database buffer for chat messages

Question

I am playing around with building a chat application using PHP and CodeIgniter.

For this, I am implementing a cache 'buffer' with memcached to hold the most recent chat messages in memory, reducing load on the database. What I want to do is this:

1. When a message arrives, I save it in memcached using the current minute (YYYY-MM-DD-HH-MM) as the key. No database I/O involved. The idea being that all messages from the same minute are collected under the same key.
2. Users receive new chat messages also fetched from memcached (for now I'm using long-polling, but this will move to WebSockets under Node.js for obvious performance reasons). Again, no database I/O involved.
3. An automated server script (cronjob) will run once every 5 minutes, collecting the memcached data from the last 5 minutes and inserting the messages into the database.
4. The memcached objects are set to go stale after 6 minutes, so we never need to keep more than 6 minutes worth of message data in memory

This for a total of one database write operation per 5 minutes and zero database read operations.

Does this sound feasible? Is there a better (maybe even built-in?) way to use memcached for this purpose?

---

Update: I have been experimenting a little now, and I have an idea for a shortcut (read: hack). I can 'buffer' the messages temporarily in the Node.js server script until I'm ready to store them. A Javascript object/array of messages in the Node.js server is basically a memory cache - kind of.

So: Every N messages/seconds, I can pass the buffered messages (the contents of the JS array) to my database, using whatever method I want, since it won't be called very often.

However, I'm worried this might cripple the Node.js server process, since it probably won't enjoy carrying around that 200 KB array.

Any thoughts on this strategy? Is it completely crazy?

Answer 1

Have you looked into HTML5 socket connections? With a socket server, you do not need to store anything. The server receives a message from one subscriber, and immediately sends it back out to the correct subscribers. I have not done this myself using HTML5, but I know the functionality now exists. I have done this before using Flash which also supports socket conenctions.

Answer 2

Why don't use INSERT DELAYED ? It offers you almost the same functionality you are trying to achieve without the need of memcached.

Anyway your solution looks good, too.