Understanding ESB [closed]

Question

though I understand what system integration is, I am a bit new to all the newest approaches. I am farily familiar with web services and JMS but I feel utterly confused by the concept of an ESB.

I have done some research but I still don't really get it. I work much better by example rather than theory.

So can someone please illustrate a simplistic example to demonstrate why one would use an Enterprise Service Bus vs just a Queue , a web service , the file system or else?

I would like the example to amplify the capabilities of the ESB which could not be achieved by any other conventional intgration method or at least not with the same efficiency.

All replies are greatly appreciated.

Thanks, Bob

Answer 1

This is going to sound a bit harsh, but basically if you needed an ESB, you'd know you needed an ESB.

For a majority of use cases, the ESB is a solution looking for a problem. It's a stack of software over engineered for most scenarios. Most folks simply do not do enough variety of processing to warrant it. The "E" for "Enterprise" is notable here.

In a simple case:

tail -F server.log | grep SEVERE >> severe.log

THAT is a trivial example of an instance of an ESB scenario.

"But that's just a UNIX command pipeline!"

Yes, exactly.

The "ESB" part is the "|" and the ">>"

The ESB is the run time within which you can link together modules, monitor traffic, design all sorts of whacky scenarios like fan outs and joins, etc. etc.

ESBs are notable for having a bunch of connectors to read a bunch of sources and write a bunch of destinations. They're notable for weaving more complicated graphs and workflows for processing using rather coarse logic blocks.

But what most folks typically do is:

input -> DO_STUFF -> output

With an ESB they get:

ESB[input -> DO_STUFF -> output]

In the wild, most pipelines simply are not that complicated. They tend to have one off logic that's not reusable, and folks tend to glob it together in to a single logic module.

Well, heck, you can do that with a Perl script.

Long pipelines in ESBs tend to be more inefficient than not. With lots of marshaling of data in to and out of generic modules (since you rarely use a binary payload).

So, say, CSV comes in, converts to XML, process it, output XML for input to another step as XML, which marshals it, works on it, converts it back in to XML for Yet Another step. Rinse and repeat until the CPU hits 400% (multi-core FTW).

Then some one comes up with "Hey, if I drag an drop these modules together in to a single routine, we skip all this XML junk!", and you end up with "input -> DO_STUFF -> output".

For large systems, with lots of web services that need to do casual, ad hoc integration, they can be fine. If you're in a business that does that a lot, they can work really well. When you have dozens of pipelines, they can help with the operational aspect of managing them.

But for complicated pipelines, if you have a lot of steps, maybe it's not such a good idea beyond prototyping, especially if there's any real volume involved. Mind, you may not having any choice, depends on the systems you're integrating.

If not, if you have a single interface you need to stand up -- then just do it. Do it in Perl, in Java, in C#, in whatever. Don't run out and spool up some odd 100MBs of infrastructure and complexity that you now to get to learn, master, and maintain.

So, again, if you needed an ESB, you'd know it. Really. You'd have whatever system you've built together of disparate stuff, you'd be fighting it, talking to colleagues about what a pain all this stuff is, and you'd stumble across some link to some site to some vendor and read a white paper and go "THAT'S IT!", but if you haven't done that yet, then you're not missing anything.

Answer 2

ESB is for the cases where you do have that web service and queue and file system all in the same system and need to integrate them. An ESB product usually solves the following

1. Security
2. Message routing
3. Orchestration (which is advanced message routing)
4. Protocol transformation
5. Message transformation
6. Monitoring
7. Eventing

You can do all of these with other tools as well and if you just need one or two of these capabilities you can probably do without and ESB (as it introduced additional complexity) but when you need several of them an integrated solution in the form of an ESB can be a better solution.