Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sharing Azure Service Fabric Reliable Queues Between Services

I'm diving into Service Fabric (from the Cloud Services world) and am hitting a few speed bumps with how ReliableQueues work.

Let's say I have 2 stateful services StatefulService1 and StatefulService2.

If I need to have StatefulService1 send a message in a queue that StatefulService2 will pick up and read am I able to use ReliableQueues or are ReliableQueues isolated within the service they are created in?

If that is the case then what is the purpose of having ReliableQueues? The usual pattern behind them is for another process to act on the messages. I understand why isolating a Dictionary to a service would make sense, but not a queue...

Is my best option to rely on a traditional approach to send this message such as a Storage Queue or does ServiceFabric offer a solution for passing message queues between services?

UPDATE

Just want to clarify that I did attempt to dequeue a message created in StatefulService1 from within StatefulService2 and it came up empty. Dequeing from within StatefulService1 worked fine as expected.

like image 939
INNVTV Avatar asked Feb 04 '23 23:02

INNVTV


2 Answers

Reliable Collections are in memory data structures that are not intended for inter-service communications. If you would like to establish a communication channel between StatefulService1 and StatefulService2, you have the following options:

  1. Use Communication Listeners. You can have custom listeners for the protocol of your choice, including HTTP, WCF or your custom protocol. You can read more about it in this section. For example, StatefulService2 can open up an HTTP endpoint that StatefulService1 can POST/GET to.

  2. Use an external queuing system, like Servicebus, EventHub or Kafka, where StatefulService1 can post events to. StatefulService2 can be consumer service that consumes events from the queue and processes it.

like image 93
Sujay S Kumar Avatar answered Mar 10 '23 12:03

Sujay S Kumar


Reliable Collections (queue and dictionary) are not intended for communication. With queues, it's a 2PC, so only one process can access it at any point in time. Note that when you use stateful services with partitions, to access the data both service instances have to be on the same partition. Different partitions cannot access the same data.

Relying on either traditional methods or implementing your own communication listener is the way to go. With the traditional way - keep in mind that you'll need to decide if you want to partition your queues just like your services are or not.

like image 27
Sean Feldman Avatar answered Mar 10 '23 11:03

Sean Feldman