How to broadcast cache invalidate messages to all servers running a web app?

Question

I have a Java based web app hosted on AWS. It is read-mostly so it makes a lot of sense to cache objects retrieved from the database for performance.

When I do update an object, I would like to be able to broadcast to all the servers that the object was saved and it should be invalidated from all local caches.

The does not need to be real time. Stale objects are annoying and need to be flushed within about 20 seconds. Users notice if they stick around for minutes. Cache invalidation does not have to happen the millisecond that objects get saved.

---

What I've thought about

• I've looked into broadcast technologies just as jGroups, but jGroups isn't supported on AWS.
• I don't think that Amazon's SQS messaging service can be made into a broadcast service.
• I'm considering using the database for this purpose: I'd write events to a database table and have each server poll this table every few seconds to get a new list items.

Answer 1

Two options come to mind. The first is to use Amazon SNS, which can use SQS as a delivery backend. This might be overkill, though, since it's designed as a frontend to lots of delivery types, including e-mail and SMS.

The approach I'd try is something along the lines of Comet-style push notifications. Have each machine with a cache open a long-lived TCP connection to the server who's responsible for handling updates, and send a compact "invalidate" message from that server to everyone who's listening. As a special-purpose protocol, this could be done with minimal overhead, perhaps just by sending the object ID (and class if necessary).