Integration architecture best practices for Enterprise applications

Question

Our application is going to integrate like a consumer to a bunch of external systems.

Most of this integrations are not just message processing and routing. There are a plenty of complex logic, like storing a current state, scheduled executions and other stuff.

Besides, each integration doesn't share much common logic.

What is the best practice to build this kind of systems?

Should I build all-in-one integration layer? It can be a monolithic application with different apache camel routes and processors for each integration:

Or should i split it to a bunch of tiny and simple standalone apps so they can be scaled and deployed independently?

What benefits and drawbacks can i get with each solution?

Answer 1

There are quite a few criteria to have in mind:

• Economy: project cost, operational cost
• Throughput: data volume per time
• Latency: travel time of messages
• Security: data protection
• Reliability: likelyhood of failures
• Flexibility: ease to react on changing requirements
• Process Support: control of data flow, event/error handling

The choice of a good solution architecture depends on the importance of such criteria for the given IT environment. For enterprise applications, it is quite common to use an integration platform rather than building the integration logic into the applications. Such a platform typically includes components for connectivity, message mapping, routing, monitoring/alerting, logging, accounting, change management, etc.

Ask your favorite search engine for Integration Patterns or Enterprise Application Integration.

Answer 2

Let me share my opinion. In general as Axel Kemper has stated there are a lot of factors that can influence your decision and there is no a silver bullet solution here.

I'll try to keep it technical, so:

A Monolithic application:

• Easier to deploy. In general you'll need only one/a couple of servers. From the point of view of developer the difference may be not that obvious, but talk to your dev-ops team and they'll immediately say that deploying one application is much easier for them

• Its easier to monitor. Basically for the same reason as above. Ask devOps about this as well:)

• It can be faster. If different Integration Apps should interconnect somehow (its not stated in your scheme, though), then potentially they can suffer from slow "over-network" protocol. In a monolithic application everything is inside the same JVM.

• Potentially requires less servers. If you're running in a private cloud/some outdated environment it can be quite a hassle to plug a new server. You can get 2-3 servers for your application and that's it :)

"Multiple integration applications" architecture (in my understanding it really resembles micro-services architecture in the integration domain) has the following advantages:

• Easier to upgrade different parts of it. If you have a monolithic application, there is no normal way to upgrade only part of the API, there should be one big upgrade for the whole application. If different "integration point" are developed by different Teams, its not obvious to coordinate between releases.

• As a result of previous bullet it will potentially take less time to fix a bug in an integration point (from the time when the bug has been detected to the time that the fix gets deployed in production). Just fix in one particular integration point and redeploy it. Its much more lightweight.

• Scales "out" better. If you experience an intensive usage of some particular integration point, you can easy add another one (or more) on different server. In a monolithic approach you'll have to install the whole application to achieve a similar effect. Another use case is that if you are deployed in a cloud and you have a client willing to pay for an exclusive access to a particular integration point.

• Easier to test (arguably). If you're running integration/system tests to check your integration point, then you can organize the CI flow so that the system will just run in parallel. Add to this a much more lightweight startup and you'll get a much more flexible flow.

I might have missed some aspects of comparison (for sure) but that's the direction IMO.

Hope this helps