How to deal with shared state in a micro-service architecture?

Question

In our company we are transitioning from a huge monolithic application to a micro-service architecture. The main technical drivers for this decision were the need to be able to scale services independently and the scalability of development - we've got ten scrum teams working in different projects (or 'micro-services').

The transition process is being smooth and we've already started to benefit from the advantages of this new technical and organizational structures. Now, on the other hand, there is a main point of pain that we are struggling with: how to manage the 'state' of the dependencies between these micro-services.

Let's put an example: one of the micro-services deals with users and registrations. This service (let's call it X) is responsible for maintaining identity information and thus is the main provider for user 'ids'. The rest of the micro-services have a strong dependency on this one. For example, there are some services responsible for user profile information (A), user permissions (B), user groups (C), etc. that rely on those user ids and thus there is a need for maintaining some data sync between these services (i.e. service A should not have info for a userId not registered in service X). We currently maintain this sync by notifying changes of state (new registrations, for example) using RabbitMQ.

As you can imagine, there are many Xs: many 'main' services and many more complicated dependencies between them.

The main issue comes when managing the different dev/testing environments. Every team (and thus, every service) needs to go through several environments in order to put some code live: continuous integration, team integration, acceptance test and live environments.

Obviously we need all services working in all these environments to check that the system is working as a whole. Now, this means that in order to test dependent services (A, B, C, ...) we must not only rely on service X, but also on its state. Thus, we need somehow to maintain system integrity and store a global & coherent state.

Our current approach for this is getting snapshots of all DBs from the live environment, making some transformations to shrink and protect data privacy and propagating it to all environments before testing in a particular environment. This is obviously a tremendous overhead, both organizationally and in computational resources: we have ten continuous integration environments, ten integration environments and one acceptance test environment that all need to be 'refreshed' with this shared data from live and the latest version of the code frequently.

We are struggling to find a better way to ease this pain. Currently we are evaluating two options:

1. using docker-like containers for all these services
2. having two versions of each service (one intended for development of that service and one another as a sandbox to be used by the rest of the teams in their development & integration testing)

None of these solutions ease the pain of shared data between services. We'd like to know how some other companies/developers are addressing this problem, as we think this must be common in a micro services architecture.

How are you guys doing it? Do you also have this problem? Any recommendation?

Sorry for the long explanation and thanks a lot!

Answer 1

This time I've read your question from different perspective, so here is a 'different opinion'. I know it may be too late but hope it helps with further development.

It looks like shared state is a result of wrong decoupling. In 'right' microservice architecture all microservices have to be isolated functionally rather than logically. I mean all three user profile information (A), user permissions (B), user groups (C) look functionally the same and more or less functionally coherent. They seem to be a single user microservice with a coherent storage. I don't see here any reasons of decoupling them (or at least you haven't told about them).

So the real problem is related to microservice isolation. Ideally each microservice can live as complete standalone product and deliver a well defined business value. When elaborating system architecture we break up it into tiny logical units (A, B, C, etc in your case, or even smaller) and then define functionally coherent subgroups. I can't tell you exact rules of how to do that, perhaps some examples. Complex communication/dependencies between the units, many common terms in their ubiquitous languages so it looks like such units belong to the same functional group and thus to microservice.

So from your example, since there is a single storage you have only way of managing its consistency as you did.

BTW I wonder what actual way you've solved your problem? Also if you like my idea fell free to accept it.