About the Mediator in Event-Driven Topology

Question

I was reading this article called Variations in event-driven architecture in which they demonstrate both the mediator and broker topologies.

According to the article the mediator topology looks somewhat like this:

The event flow starts with the client sending an event to an event queue, which is used to transport the event to the mediator. The event mediator receives the initial event and orchestrates that event by sending additional asynchronous events to event channels to execute each step of the process. Event processors, which listen on the event channels, receive the event from the even mediator and execute specific business logic to process the event [...] It is important to note that the event mediator doesn't actually perform the business logic necessary to process the initial event, rather, it knows of the steps required to process the event [...] The event channels can be either message queues o message topics.

So, I was studying this diagram, trying to understand how the mediator could determine when a given processor has finished processing a given event, such that it could orchestrate the next step of the process.

The article is not clear enough when it says

For each initial event step, the event mediator creates a processing event, sends that processing event and waits for the processing event to be processed by the corresponding event processor. This process continues until all of the steps in the initial event have been processed.

Now, the article is clear in that the communication is asynchronous, and event messages will travel through message queues, but the diagram does not show any events coming out of the event processor and back to the mediator.

The article says the mediator waits for the event processor to finish, but it is not clear how this is supposed to happen in architectural terms.

Is it asynchronous, queue-based RPC (e.g. Rabbit RPC), or is there another listener waiting for an asynchronous response somewhere?

Any thoughts on how this can be implemented from an architectural standpoint?

Answer 1

It seems to me that they just didn't draw the events returning from the event processors, perhaps because they may not specifically be events (like some sort of a callback) or because they may not be normal events (perhaps events that only go back to the mediator and are not visible to any other subscribers), depending on what you use as your mediator. This part seems to indicate something like that:

The event mediator can be implemented in a variety of ways. Understand each implementation option to ensure that the solution you choose for the event mediator matches your needs.

The simplest and most common implementation of the event mediator is through open source integration hubs such as Spring Integration, Apache Camel, or Mule ESB. Event flows in these open source integration hubs are typically implemented through Java code or a DSL (domain-specific language). For more sophisticated mediation and orchestration, you can use BPEL (business process execution language) coupled with a BPEL engine such as the open source Apache ODE. BPEL is a standard XML-like language that describes the data and steps required for processing an initial event. For very large applications requiring much more sophisticated orchestration (including steps involving human interactions), you can implement the event mediator using a business process manager (BPM) such as jBPM.

Understanding your needs and matching them to the correct event mediator implementation is critical to the success of any event-driven architecture using this topology. Using an open source integration hub to do very complex business process management orchestration is a recipe for failure, just as is implementing a BPM solution to perform simple routing logic.

They mentioned Spring as a possible implementation - I've never used it, but looking at the documentation (here) I see the concept of a reply channel:

...when the service method returns a non-null value, the endpoint will attempt to send the reply message to an appropriate reply channel.

---

The goal is to send one or more messages off to be processed asynchronously, then to send other messages off when the results come back. I don't think it matters too much at the pattern level exactly how those results come back (function callback, 'response' event, web API call, whatever) - that will depend on your specific infrastructure.

To me it sounds quite a bit like the Saga pattern (link). In that, the Saga (or Mediator) knows the steps needed to complete some task and maintains some state about the progress of that task.

It fires off commands to be processed and listens for responses. When a response (event) comes in, it updates its state, then uses its state to determine what command(s) need to be fired off next.

That continues until either A) the process is completed, or B) a step in the process fails (in which case it might reverse direction and start firing compensating commands to 'undo' the prior actions).

Using the diagram below from the post your referenced, the "thoughts" of the saga/mediator might be along the lines of...

1. Relocation event, so fire off the Change Address command and wait.
2. AddressChanged event received. Now I've changed the address, but I haven't recalced the quote or updated the claims, so I'll send those off (they don't conflict, so send them both) and wait.
3. ClaimsUpdated event received. Still need the recalc before moving forward, so keep waiting.
4. QuoteRecalced event received. Now that I've updated the address, recalced the quote, and updated the claims, I can send off the Adjust Claims command, and wait.

... and so on.

You would maybe want to add persistence (so it can pick up where it left off in case of a crash), idempotent event processors (so replay of events doesn't cause issues), and/or timeouts (to not get stuck waiting forever if a response event gets missed/lost).

Answer 2

I think you have the answer on the diagram itself:

Initial Event step (red color) is the key. Every Event Processor produces an event, which is what gets into the Event Queue and then to the Event Mediator.

The architecture is Event Driven and asynchronous. Single Event Queue handles path to the Event Mediator. And since this is the only way to get the event into the Event Mediator, obviously, anything wanting to send an event to the mediator would need to use this path.

At some point, after certain event, the Event Mediator will declare the operation as successfully complete and will not dispatch more events to the Event Processors.

Although, I must say, you are right, this is not clearly stated in the article. I assume this will be better clarified in the book they are previewing.