WCF Pub/Sub with subscriber caching

Question

Problem: how to provide a distributed, scalable and disaster resistant pub/sub service with WCF.

Details:

Note that this approach is being considered in addition to messaging/middleware solutions such as Tibco EMS.

I've been looking into WCF, particularly how it may be used to offer pub/sub. On this subject this article is very good: WCF pub-sub.

In the article the author attempts to tackle the problem of having multiple publishers (as one would have with a service layer scaled across several boxes). The problem being that if client A registers with Publisher A but Publisher B wishes to publish an event, then publisher B won't know about client A. i.e. no one told publisher B that client A wanted to be notified about events. The author suggests a pub/sub service as a solution. The pub/sub service would centrally store subscriptions. However, if I wanted to make the pub/sub service disaster resistant by having a secondary/dual pub/sub service then I have the same original problem.

So, I think there are a couple of solutions to the problem:

1. Store subscriber details in a distributed cache (see questions: q1 and q2).
2. Store subscriber details in a database/central file system.

Can anyone think of any other solutions (i.e. I've not missed some fantastic magical feature of WCF?) Any comments appreciated.

Answer 1

I had the same problem and I did a lot of research on the issue. The problem is actually simple. You want to keep some centralized state, but in distributed way. I found that the best way to achieve this is by using a distributed cache. Look at velocity for example. There is no native WCF solution that I know that can solve the state management issue. I have even looked into durable services, where state management is handled by WCF, however not suitable for a pub/ sub service, because the state needs to be centralized for all client connections. Storing data in a database is also an option, but the cost is the need for a database, and even with a database you can have a single point of failure if the database is not clustered accorss multiple machines.

At the end, I figured it is actually expensive to implement something with zero points of failure and if you do decide to go there then take a look at Azure, the future of storage is on the cloud, Azure services will be fully scalable and distributed, but we are not there yet.