Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How should one use Disruptor (Disruptor Pattern) to build real-world message systems?

As the RingBuffer up-front allocates objects of a given type, how can you use a single ring buffer to process messages of various different types?

You can't create new object instances to insert into the ringBuffer and that would defeat the purpose of up-front allocation.

So you could have 3 messages in an async messaging pattern:

  1. NewOrderRequest
  2. NewOrderCreated
  3. NewOrderRejected

So my question is how are you meant to use the Disruptor pattern for real-world messageing systems?

Thanks

Links: http://code.google.com/p/disruptor-net/wiki/CodeExamples

http://code.google.com/p/disruptor-net

http://code.google.com/p/disruptor

like image 341
JamesRedcoat Avatar asked Aug 03 '11 21:08

JamesRedcoat


People also ask

What is the disruptor pattern?

Disruptor has an array based circular data structure (ring buffer). It is an array that has a pointer to next available slot. It is filled with pre-allocated transfer objects. Producers and consumers perform writing and reading of data to the ring without locking or contention.

How does LMAX Disruptor work?

The disruptor pattern is a batching queue backed up by a circular array (i.e. the ring buffer) filled with pre-allocated transfer objects which uses memory-barriers to synchronize producers and consumers through sequences.


1 Answers

One approach (our most common pattern) is to store the message in its marshalled form, i.e. as a byte array. For incoming requests e.g. Fix messages, binary message, are quickly pulled of the network and placed in the ring buffer. The unmarshalling and dispatch of different types of messages are handled by EventProcessors (Consumers) on that ring buffer. For outbound requests, the message is serialised into the preallocated byte array that forms the entry in the ring buffer.

If you are using some fixed size byte array as the preallocated entry, some additional logic is required to handle overflow for larger messages. I.e. pick a reasonable default size and if it is exceeded allocate a temporary array that is bigger. Then discard it when the entry is reused or consumed (depending on your use case) reverting back to the original preallocated byte array.

If you have different consumers for different message types you could quickly identify if your consumer is interested in the specific message either by knowing an offset into the byte array that carries the type information or by passing a discriminator value through on the entry.

Also there is no rule against creating object instances and passing references (we do this in a couple of places too). You do lose the benefits of object preallocation, however one of the design goals of the disruptor was to allow the user the choice of the most appropriate form of storage.

like image 60
Michael Barker Avatar answered Sep 22 '22 19:09

Michael Barker