Should we introduce BizTalk/ESB?

Question

My company are about to implement a new architecture in which we have proposed BizTalk (we are a Microsoft shop) as the Enterprise Service Bus (ESB) in a SOA (please don't quote Service Oriented Ambiguity) environment.

Our business is to take Orders through our new Order Capture GUI which must connect to our Customers Database, Product Catalogue, Ordering System and some other ancillary systems each of which will be exposed as WCF services, orders are then passed to our Order Management and other downstream systems for fulfilment and finally to our Billing system for invoicing. Currently each system has its own GUI and uses manual process to pass information between them, in an effort to automate and integrate the natural thought was to introduce an ESB to connect them.

Some of my rationale for an ESB is, the bus will worry about how to connect the systems (each system is agnostic and knows nothing of any other system) and how to format/translate the information. It is highly likely that in the future some of the existing systems will be swapped out for new systems or systems within our family of companies.

This seems to make sense to me but I am now being met with some resistance as to why introduce it when a Point-to-Point solution could suffice.

Unfortunately in the company history (prior to my appointment) an initial attempt to introduce BizTalk failed, but I am confident that it has a place and I can deliver it.

My question is perhaps not so much about BizTalk but whether an ESB is a good idea in my scenario described, when does it make sense to introduce an ESB?

Answer 1

Okay. ESB Guidance on Biztalk from the presrciptive architechture group - http://msdn.microsoft.com/en-us/library/cc487894.aspx

We use BizTalk where I work to do a lot of things. He have some simple point intgerations. We have some more complicated point integrations with hihgly customized adpaters and pipelines. We have divisional enterprise system integrations for customer master, product info and price and quote to order. These are all seperate BizTalk applications. Some quite small and some quite large. We mainly have used BizTalk to do point to point/many point slutions without using an ESB pattern. The implementation of an ESB implies a level of governance of the bus itself and the enterpise message standars that will be permitted on the bus. If you will be interfacing with a large number of systems with a large number of different formats - ESB makes a lot of sense. If the integrations you want to achieve are less ambitious, an ESB may be overkill. That being said, it's a clean and extensible architechture. You'll have to make the cost value decision.

BizTalk is also a complicated beast but with allof the moving parts comes a level of flexibility that is wonderful. But be prepared for a learning curve or some consultant costs.

Answer 2

I just got asked this same question by a colleague and this is what I said to him:

In most integration scenarios you can go quite far before using something like BizTalk. I would make sure that I couldn’t do the integration more simply before going with BizTalk.

Only if it seems that the integration solution needs to scale to high volumes with low latency (it’s got a fantastic asynchronous publish-subscribe mechanism), and you need fault tolerance (it’s got redundancy, scaling and message retry features) and governance over the solution (it’s got Business Activity Monitoring) would you really have strong arguments to consider using BizTalk. And if you know that there are multiple future integrations that will be needed then it gets really compelling to use BizTalk.

On the other hand you need to make sure the skills are available to operate the thing. It takes a while to learn and a paradigm shift for the developers of the systems. However its built from the ground up in .NET and SQL Server so there is quite a lot of familiarity in the tooling and concepts.

I think the key thing is to get the conceptual architecture of a solution right taking into account the non-functional requirements like performance, availability, extensibility, resilience, robustness, and scalability and making sure they are properly addressed by the technical design. You may find its cheaper to pay the 35k$ per CPU license for what BizTalk gives you out the box than to develop for all these NFRs.

Also I've recently implemented the new ESB Toolkit 2.0 at a client and am very happy with it. The Itinerary (see the Routing Slip pattern http://www.enterpriseintegrationpatterns.com/RoutingTable.html) processing functionality really makes composing web services easy, flexibly and fast.