Please, imagine you have a method like the following:
public void PlaceOrder(Order order)
{
this.SaveOrderToDataBase(order);
this.bus.Publish(new OrderPlaced(Order));
}
After the order is saved to the database, an event is published to the message queuing system, so other subsystems on the same or another machine can process it.
But, what happens if this.bus.Publish(new OrderPlaced(Order))
call fails? Or the machine crashes just after saving the order into the database? The event is not published and other subsystems cannot process it. This is unacceptable. If this happens I need to ensure that the event is eventually published.
What are the acceptable strategies can I use? Which is the best one?
NOTE: I don't want to use distributed transactions.
EDIT:
Paul Sasik is very close, and I think I can achieve 100%. This is what I thought:
first create a table Events in the database like the following:
CREATE TABLE Events (EventId int PRIMARY KEY)
You may want to use guids instead of int, or you may use sequences or identities.
Then do the following pseudocode:
open transaction
save order and event via A SINGLE transaction
in case of failure, report error and return
place order in message queue
in case of failure, report error, roll back transaction and return
commit transaction
All events must include EventId. When event subscribers receive an event, they first check EventId existence in database.
This way you get 100% realiability, not only 99.999%
Message Queuing (MSMQ) Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. Applications send messages to queues and read messages from queues.
But the point still remains: Your event stream is a message queue. As always, context is king. Requirements and many other factors will play into how you want to handle messaging.
There are many aspects to consider when choosing a message queue, such as propagation, delivery, persistence, and consumer groups. In this section, I will explain them briefly. The propagation means how messages are transferred by the message queue. There are 2 types of propagation, One-to-one is quite simple.
This allows applications to be developed for the Internet as well, including both server-side (Internet Information Server) and client-side (Internet Explorer) applications. The .NET Framework offers a set of managed Message Queuing objects.
The correct way to ensure the event is eventually published to the message queuing sytem is explained in this video and on this blog post
Basically you need to store the message to be sent into the database in the same transaction you perform the bussines logic operation, then send the message to the bus asynchronously and delete the message from the database in another transaction:
public void PlaceOrder(Order order)
{
BeginTransaction();
Try
{
SaveOrderToDataBase(order);
ev = new OrderPlaced(Order);
SaveEventToDataBase(ev);
CommitTransaction();
}
Catch
{
RollbackTransaction();
return;
}
PublishEventAsync(ev);
}
async Task PublishEventAsync(BussinesEvent ev)
{
BegintTransaction();
try
{
await DeleteEventAsync(ev);
await bus.PublishAsync(ev);
CommitTransaction();
}
catch
{
RollbackTransaction();
}
}
Because PublishEventAsync may fail you have to retry later, so you need a background process for retrying failed sendings, something like this:
foreach (ev in eventsThatNeedsToBeSent) {
await PublishEventAsync(ev);
}
You can make the this.bus.Publish
call part of a database transaction of the this.SaveOrderToDataBase
. This means that this.SaveOrderToDataBase
executes in transaction scope and if the db call fails you never call the mq and if the mq call fails then you roll back the db transaction leaving both systems in a consistent state. If both calls succeed you commit the db transaction.
Pseudocode:
open transaction
save order via transaction
in case of failure, report error and return
place order in message queue
in case of failure, report error, roll back transaction and return
commit transaction
You didn't mention any specific db technology so here's a link to a wiki article on transactions. Even if you're new to transactions, it's a good place to start. And a bit of good news: They are not hard to implement.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With