EventStore and application lifecycle

Question

Maybe this question is silly, but I'm a little confused about. Let's suppose we want leverage this pattern:

• what exactly is the event storage scope in an enterprise application?

• Does an event storage share among multiple processes, or is just an in-process concept?

• What happens to events when the application close? Are they bound to an application "instance" or to an application?

• What are the differences between an event storage and a MessageBus with Publisher/Subscriber ( a part the fact we can store the message history?

• Who's responsible for the message idempotence?

• what does this sentence actually mean:"Interestingly enough, even without the presence of distributed transactions across the various resources involved, such as a message queue and persistent storage, the EventStore is able to ensure a fully transactional experience. This is achieved by breaking apart a distributed transaction into smaller pieces and performing each one individually" ( from this project ) I can't understand how breaking a transaction in several small pieces, even if all are transactional per-se can replace a distributed transaction.

Answer 1

what exactly is the event storage scope in an enterprise application?

The event store is just like a database in this sense. It isn't scoped by any given application. It can be scoped by business/linguistic boundaries however. For example if you partition your system into sub-systems, each sub-system can have its own instance of an event store.

Does an event storage share among multiple processes, or is just an in-process concept?

It is not an in-process concept. It is shared among processes/applications just like a database would be.

What happens to events when the application close? Are they bound to an application "instance" or to an application?

The event store would store all events that the application asked it to store. Events are keyed by a stream ID which is usually the ID of an aggregate root. This isn't bound to specific application instances.

What are the differences between an event storage and a MessageBus with Publisher/Subscriber ( a part the fact we can store the message history?

Storage of message history is basically the central difference in terms of functionality. The difference in use case is that a message bus is used to pass messages between endpoints where as an event store is used to persist messages (usually events).

Who's responsible for the message idempotence?

You as the developer. The event store views events as serialized data keyed by a stream, possibly with versioning. With versioning it can handle certain conflicts, but it is up to you to determine whether a message is idempotent or not.

I can't understand how breaking a transaction in several small pieces, even if all are transactional per-se can replace a distributed transaction.

Take a look at Clarifying the Saga pattern. The basic idea is that instead of grouping multiple operations under a single distributed transaction, the operation is broken into parts. If a certain part fails (which would cause a rollback in a distributed tx) an error message is sent which can allow interested parties to rollback operations. This can be seen as a form of compensation and it is a more natural way to analyse many business scenarios. For example, when a payment transaction is deemed invalid, it isn't deleted but a compensating transaction is added. This way of representing activity is in better alignment with reality because in reality rarely is something "undone".

Answer 2

How many questions!

what exactly is the event storage scope in an enterprise application?

Event storage is not properly a pattern, it's a technique usually used with two different (but strongly related) patterns: Event Sourcing and Command and Query Responsibility Segregation. Being a "storage", it's just a way to persist the state of the application that is relevant to the business.

Both patterns are often used in conjunction with a domain model, since they work well with the patterns introduced by Evans in Domain Driven Design.

An EventStore allows you to persist domain events (the Event Sourcing way) or application events (aka, Commands, in the CQRS one). It differs from document and relational storage since you don't save the state of the model but the events that led to it. However you could use either a RDBMS or a document db to store events.

Then to retrieve an entity, you can simply play forward each of events/commands registered, in sequence. Snapshots can be used to make this process faster.

Does an event storage share among multiple processes, or is just an in-process concept?

It depends on the store implementation, but there's no reason that prevents it's use among multiple process and/or applications.

What happens to events when the application close? Are they bound to an application "instance" or to an application?

Again it depends on the store implementation. The simplest possible event store save events into numbered files, thus when the application shuts down the events are still there (this always reminds me of Thompson's words: "we have persistent objects, we call them files").
But nothing prevents you from having a volatile event store, just in memory, if your application really needs it. I would implement it as an append-only collection keeping entry order.

What are the differences between an event storage and a MessageBus with Publisher/Subscriber ( a part the fact we can store the message history?

A message bus is a tool to deliver messages. Events (and commands) are messages, thus you can use it to deliver them. An event store, instead, is a persistence tool.

Who's responsible for the message idempotence?

In most common scenarios, the guy that design the domain model. In a non-DDD system, it's the guy that design the messages (events or commands). Indeed, idempotence must be guaranteed by the receivers of the messages, not by the technology per-se.

Given that, an EventStore could merge duplicated messages when it detect them. But this doesn't make, per-se, the model idempotent.

what does this sentence actually mean:"Interestingly enough, even without the presence of distributed transactions across the various resources involved, such as a message queue and persistent storage, the EventStore is able to ensure a fully transactional experience. This is achieved by breaking apart a distributed transaction into smaller pieces and performing each one individually" ( from this project ) I can't understand how breaking a transaction in several small pieces, even if all are transactional per-se can replace a distributed transaction.

It depends on the meaning that the author assign to "fully transactional experience". To me, this sentence looks wrong, since it would break the Brewer's theorem.

You could find useful this CQRS Journey from Microsoft and Greg Young.

See you tomorrow, at the office :-)