Design choice for a microservice event-driven architecture

Question

Let's suppose we have the following:

DDD aggregates A and B, A can reference B.

A microservice managing A that exposes the following commands:

• create A
• delete A
• link A to B
• unlink A from B

A microservice managing B that exposes the following commands:

• create B
• delete B

A successful creation, deletion, link or unlink always results in the emission of a corresponding event by the microservice that performed the action.

What is the best way to design an event-driven architecture for these two microservices so that:

1. A and B will always eventually be consistent with each other. By consistency, I mean A should not reference B if B doesn't exist.
2. The events from both microservices can easily be projected in a separate read model on which queries spanning both A and B can be made

Specifically, the following examples could lead to transient inconsistent states, but consistency must in all cases eventually be restored:

Example 1

• Initial consistent state: A exists, B doesn't, A is not linked to B
• Command: link A to B

Example 2

• Initial consistent state: A exists, B exists, A is linked to B
• Command: delete B

Example 3

• Initial consistent state: A exists, B exists, A is not linked to B
• Two simultaneous commands: link A to B and delete B

I have two solutions in mind.

Solution 1

• Microservice A only allows linking A to B if it has previously received a "B created" event and no "B deleted" event.
• Microservice B only allows deleting B if it has not previously received a "A linked to B" event, or if that event was followed by a "A unlinked from B" event.
• Microservice A listens to "B deleted" events and, upon receiving such an event, unlinks A from B (for the race condition in which B is deleted before it has received the A linked to B event).

Solution 2:

• Microservice A always allows linking A to B.
• Microservice B listens for "A linked to B" events and, upon receiving such an event, verifies that B exists. If it doesn't, it emits a "link to B refused" event.
• Microservice A listens for "B deleted" and "link to B refused" events and, upon receiving such an event, unlinks A from B.

EDIT: Solution 3, proposed by Guillaume:

• Microservice A only allows linking A to B if it has not previously received a "B deleted" event.
• Microservice B always allows deleting B.
• Microservice A listens to "B deleted" events and, upon receiving such an event, unlinks A from B.

The advantage I see for solution 2 is that the microservices don't need to keep track of of past events emitted by the other service. In solution 1, basically each microservice has to maintain a read model of the other one.

A potential disadvantage for solution 2 could maybe be the added complexity of projecting these events in the read model, especially if more microservices and aggregates following the same pattern are added to the system.

Are there other (dis)advantages to one or the other solution, or even an anti-pattern I'm not aware of that should be avoided at all costs? Is there a better solution than the two I propose?

Any advice would be appreciated.

Answer 1

Microservice A only allows linking A to B if it has previously received a "B created" event and no "B deleted" event.

There's a potential problem here; consider a race between two messages, link A to B and B Created. If the B Created message happens to arrive first, then everything links up as expected. If B Created happens to arrive second, then the link doesn't happen. In short, you have a business behavior that depends on your message plumbing.

Udi Dahan, 2010

A microsecond difference in timing shouldn’t make a difference to core business behaviors.

A potential disadvantage for solution 2 could maybe be the added complexity of projecting these events in the read model, especially if more microservices and aggregates following the same pattern are added to the system.

I don't like that complexity at all; it sounds like a lot of work for not very much business value.

Exception Reports might be a viable alternative. Greg Young talked about this in 2016. In short; having a monitor that detects inconsistent states, and the remediation of those states, may be enough.

Adding automated remediation comes later. Rinat Abdullin described this progression really well.

The automated version ends up looking something like solution 2; but with separation of the responsibilities -- the remediation logic lives outside of microservice A and B.