Pub/sub while maintaining an access to the full state of the system

Question

I have a server which maintains a global state of an application.

Clients can connect to the server and get messages about changes in the global state (publish/subscribe mechanism in order for the server to broadcast the information).

However, at start-up, the clients have no information at all about the global state, and they need it. What I want is for any new client subscribing to the system, the first notification message is the full state of the application. Then they only receive changes about this state:

1. a client connects to the messaging system
2. it subscribes to the messaging system
3. the first message it gets is the full state of the system
4. then, it only receives changes about the global state

The idea is similar to a multiplayer game where a new player must first get the full state of the game and then only changes in the game are sent.

Messaging systems like ActiveMQ or Stomp are fine for my needs, as they are multi-languages and can be used with several transport layers. However there is not this notion of sending the full state (or cumulate the last changes in a coherent way).

Of course, I could easily provide this state in a static way (first I get the full state, and then I subscribe to the publish/subscribe sytem), however I must be careful about possible changes in the meantime (did I lose some changes while processing the full state? Is this change already taken into account in the global state I just retrieved? ...). However I lose the multi-language/multi-transport aspect already provided by Stomp and ActiveMQ.

Are there some existing libraries/tools to do that? Some kind of extensions for ActiveMQ? Something similar to Stomp? Or does it need to be hand made?

Answer 1

This is not a messaging domain problem, but a specific use case. Pub/Sub is not designed to know any state information about the producer/consumer, it just proxies messages.

That said, there are a few different redelivery patterns that can be used to bridge the gap a bit (Last Image Subscription Policy, etc), but I'd opt for simpler solutions. Also, you might consider using something like Apache Camel to sit in between producers and subscribers to provide this extra logic..

As you said, the tricky part is to keeping incremental updates in sync with the full system image retrieved. Off the top, here is what I'd do...

• provide a REST service to allow any clients to pull the full state of the system
• add an incrementing version number to both the full state data and incremental update data
• when a client comes online
  • subscribe to start getting incremental updates (queue them internally for now)
  • use the REST service to get the full system state
  • then, start processing incremental updates and disregard old ones based on the version numbers