Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Best way to ensure an event is eventually published to a message queuing sytem

Please, imagine you have a method like the following:

public void PlaceOrder(Order order)
{
     this.SaveOrderToDataBase(order);
     this.bus.Publish(new OrderPlaced(Order));    
}

After the order is saved to the database, an event is published to the message queuing system, so other subsystems on the same or another machine can process it.

But, what happens if this.bus.Publish(new OrderPlaced(Order)) call fails? Or the machine crashes just after saving the order into the database? The event is not published and other subsystems cannot process it. This is unacceptable. If this happens I need to ensure that the event is eventually published.

What are the acceptable strategies can I use? Which is the best one?

NOTE: I don't want to use distributed transactions.

EDIT:

Paul Sasik is very close, and I think I can achieve 100%. This is what I thought:

first create a table Events in the database like the following:

CREATE TABLE Events (EventId int PRIMARY KEY)

You may want to use guids instead of int, or you may use sequences or identities.

Then do the following pseudocode:

open transaction
save order and event via A SINGLE transaction
in case of failure, report error and return
place order in message queue
in case of failure, report error, roll back transaction and return
commit transaction

All events must include EventId. When event subscribers receive an event, they first check EventId existence in database.

This way you get 100% realiability, not only 99.999%

like image 925
Jesús López Avatar asked Jun 11 '15 12:06

Jesús López


People also ask

What is Message Queuing (MSMQ)?

Message Queuing (MSMQ) Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. Applications send messages to queues and read messages from queues.

Is your event stream a message queue?

But the point still remains: Your event stream is a message queue. As always, context is king. Requirements and many other factors will play into how you want to handle messaging.

What are the aspects to consider when choosing message queue?

There are many aspects to consider when choosing a message queue, such as propagation, delivery, persistence, and consumer groups. In this section, I will explain them briefly. The propagation means how messages are transferred by the message queue. There are 2 types of propagation, One-to-one is quite simple.

Why do we need message queuing in the Internet?

This allows applications to be developed for the Internet as well, including both server-side (Internet Information Server) and client-side (Internet Explorer) applications. The .NET Framework offers a set of managed Message Queuing objects.


2 Answers

The correct way to ensure the event is eventually published to the message queuing sytem is explained in this video and on this blog post

Basically you need to store the message to be sent into the database in the same transaction you perform the bussines logic operation, then send the message to the bus asynchronously and delete the message from the database in another transaction:

public void PlaceOrder(Order order)
{
     BeginTransaction();
     Try 
     {
         SaveOrderToDataBase(order);
         ev = new OrderPlaced(Order);
         SaveEventToDataBase(ev);
         CommitTransaction();
     }
     Catch 
     {
          RollbackTransaction();
          return;
     }

     PublishEventAsync(ev);    
}

async Task PublishEventAsync(BussinesEvent ev) 
{
    BegintTransaction();
    try 
    {
         await DeleteEventAsync(ev);
         await bus.PublishAsync(ev);
         CommitTransaction();
    }
    catch 
    {
         RollbackTransaction();
    }

}

Because PublishEventAsync may fail you have to retry later, so you need a background process for retrying failed sendings, something like this:

foreach (ev in eventsThatNeedsToBeSent) {
    await PublishEventAsync(ev);
}
like image 175
Jesús López Avatar answered Oct 06 '22 14:10

Jesús López


You can make the this.bus.Publish call part of a database transaction of the this.SaveOrderToDataBase. This means that this.SaveOrderToDataBase executes in transaction scope and if the db call fails you never call the mq and if the mq call fails then you roll back the db transaction leaving both systems in a consistent state. If both calls succeed you commit the db transaction.

Pseudocode:

open transaction
save order via transaction
in case of failure, report error and return
place order in message queue
in case of failure, report error, roll back transaction and return
commit transaction

You didn't mention any specific db technology so here's a link to a wiki article on transactions. Even if you're new to transactions, it's a good place to start. And a bit of good news: They are not hard to implement.

like image 2
Paul Sasik Avatar answered Oct 06 '22 16:10

Paul Sasik