Service Oriented Architecture and Loose Coupling vs SQL JOINS

Question

Let's suppose we have got a SOA infrastructure like the one painted below and that every service can run on a different host (this is especially valid for the two extra-net service "web site" and "payment system").

Clearly we have got a data (persistence) layer. Suppose it's implement through EJB + JPA or something alike.

If we want to join data (in user UI) between the different services I see at least a couple of alternatives:

• we want to do efficient JOINs at RDBMS level so we have a package (ie. persistence.package) that contains all the entities and session facades (CRUD implementation) which in some way has to be shared (how ?) or deployed for every service. That said, if I change something in the order schema I must redeploy this packages introducing tight coupling between pretty much everything. Moreover the database must be unique and shared.

• to avoid such issues, we keep an entity package for each different service (i.e. order.package) and let the services communicate through some protocol (soap, rest, esb, etc.). so we can keep data locally in each host (share nothing architecture) and we don't need to redeploy the entity package. But this approach is terrible for data-mining as a query that must search and return correlated data between multiple services will be very inefficient (as we cannot do SQL joins)

Is there a better / standard approach to the issues pointed above ?

Answer 1

The main motivation for SOA is independent components that can change separately. A secondary motivation,as Marco mentioned, is simplifying a system into smaller problems that are easier to solve. The upside of different services is flexibility the downside is more management and overhead - that overhead should be justified by what you get back - see for example a SOA anti-pattern I published called Nanoservices which talks about this balance

Another thing to keep in mind is that a web-service API does not automatically mean that that's a service boundary. Several APIs that belong to a larger service can still connect to the same database underneath. so for example, if in your system payments and orders belong together you shouldn't separate them just because they are different APIs (In many systems these are indeed different concerns but, again, that's not automatic)

When and if you do find the separation into services logical than you should follow Marco's advice and ensure that the services are isolated and don't share databases. Having services isolated this way serves toward their ability to change. You can then integrate them in the UI with a composite front end. You should note that this works well for the operational side of the application as there you only need a few items from each service. For reporting you'd want something like aggregated reporting i.e. export immutable copies of data into a central database optimized for reporting (e.g denormalized star-schema etc.)

Answer 2

Oh my friend you're just complicating the whole scenario, but it is not your fault, companies like MSFT, Oracle and other big vendors of enterprise class software like to make a big picture of something that is way easier, and they do it for a reason: scaling services vertically means more licenses.

You can take a different approach and forget for a moment about all those big words, EJB, JPA... and like someone smart once said, split the big problem in smaller parts so that rather than having a big problem you have a couple of small problems which in theory should be easier to deal with.

So you have a couple of services in your system: people, payment system, orders, products, ERP... for a moment lets think that those boundaries are right in terms of business entities. Imagine those services are different physical departments of you company, which means that they only care with the data that belongs to them, nothing else.

You could then say that Payments department has its own database, the same applies to Orders, of course they still need to communicate with each other as all departments do, and that can be made easy with a system generated public surrogate key. What this all means is that each service maintains the referential integrity of all its internal entities using internal keys, but if records need to my made available to other services you can for example use a Guid key, e.g.:

The payments service needs the order ID and the Customer ID, but those entities belong to their own services, so instead of sharing a private key (primary key) of each record, each record will have instead a primary key and a surrogate external key the services will use to share among them. The thing is, you should aim to build loose coupled services, each with its own "small" database. Each service should also have each own API, which should be used not only by the front end, but by the other services as well. Another thing you should avoid is using DTC or other transaction management provider as a service wide transaction guarantor, it is something that can be archive easily with just a different architecture approach.

Anyway, read, if you haven't already about DDD, it will give you a different overview on how to build enterprise class software, and btw EJB, run away from them.

UPDATE:

You could use something like event SOA, but lets keep things simple here. A registered client comes to your site to place an order. The service responsible for this is the Orders. A list of external IDs for the products is submitted to the Orders service which then register the order, at this point the order is in a "awaiting payment" status and this service returns a public Guid order ID. For the order to be completed the customer needs to pay for the goods. The payment details are submitted to the payment's service which tries to process a payment, but for that it needs the order details because the only thing the frontend sent was the order id, and to do that it uses the GetOrderDetails(Guid orderId) from the order´s API. Once the payment is completed the Payments service calls yet another method of the Order´s API PaymentWasCompletedForOrder(Guid orderID). Let me know if there is anything you are not getting.