Use Mongoose-Transactions over multiple databases

Question

I am creating a Node.js API consisting of multiple Microservices.

Each Microservice is responsible for one or more features of my application. However, my data is structured into multiple databases which each have multiple collections.

Now I need one sevice to perform atomic operations across multiple databases. If everything happened in the same database, I'd use a normal transaction. However, I don't know how to do this with multiple databases or if this is even possible?

Example:

One of the Microservices takes care of creating users. A user must be created inside two databases. However, this should happen atomically, i.e. if the user is created, it must be created in both databases.

UPDATE: MongoDB's official docs state the following:

With distributed transactions, transactions can be used across multiple operations, collections, databases, documents, and shards.

I haven't found anything on how to perform distributed transactions with mongoose though.

I would be extremely glad if someone could give me some clarification on this topic.

Answer 1

You need to use the SAGA pattern of the microservice architecture.

The SAGA pattern is divided into two types:

1. Choreography-based saga
2. Orchestration-based saga

If you want to manage distributed transactions from a single service, then you can use Orchestration-based saga (2).

So with this pattern, you can implement a distributed transaction that either executes a chain of actions or rolls back along the chain, using compensating transactions.

I also recommend studying the patterns of microservice architecture on this site and recommend the book.

Answer 2

EDIT: Mongoose support Distributed Transactions, because it's a client to MongoDB Server. Form Mongoose point of view, a distributed transaction is just a transaction.

According to this video, on Distributed Transactions in MongoDB the Distributed Transactions is defined above the level of mongoose, and can use it.

in the documentation of mongodb, they say:

Distributed Transactions and Multi-Document Transactions Starting in MongoDB 4.2, the two terms are synonymous. Distributed transactions refer to multi-document transactions on sharded clusters and replica sets. Multi-document transactions (whether on sharded clusters or replica sets) are also known as distributed transactions starting in MongoDB 4.2.

Here is how I would try to solve this (Divide-and-conquer):

1. Try simple example of Distributed Transactions with MongoDB
2. Then try using simple mongoose with Transactions (it might be that there is be no different between , Distributed Transactions and non- Distributed Transactions as far as mongoose knows, because the Transactions is in higher level – see video).
3. Then try to combine the 2 solutions and see it this works,
4. If does not work with mongoose, I would try to implement Distributed Transactions with MongoDB, as the video implay that they spent a lot of effort in this, and since mongoose just let you do things that you can also do with MongoDB alone. Moving from mongoose to MongoDB maybe not so simple, but implementing Distributed Transactions is very hard.