Apache camel to aggregate multiple REST service responses

Question

I m new to Camel and wondering how I can implement below mentioned use case using Camel,

We have a REST web service and lets say it has two service operations callA and callB. Now we have ESB layer in the front that intercepts the client requests before hitting this actual web service URLs.

Now I m trying to do something like this - Expose a URL in ESB that client will actually call. In the ESB we are using Camel's Jetty component which just proxies this service call. So lets say this URL be /my-service/scan/

Now on receiving this request @ESB, I want to call these two REST endpoints (callA and callB) -> Get their responses - resA and resB -> Aggregate it to a single response object resScan -> return to the client.

All I have right now is -

<route id="MyServiceScanRoute">
<from uri="jetty:http://{host}.{port}./my-service/scan/?matchOnUriPrefix=true&amp;bridgeEndpoint=true"/>
<!-- Set service specific headers, monitoring etc. -->  
<!-- Call performScan -->
<to uri="direct:performScan"/>
</route>

<route id="SubRoute_performScan">
<from uri="direct:performScan"/>
<!--  HOW DO I??
Make callA, callB service calls. 
Get their responses resA, resB.
Aggregate these responses to resScan
 -->
</route>

Answer 1

I think that you unnecessarily complicate the solution a little bit. :) In my humble opinion the best way to call two independed remote web services and concatenate the results is to:

• call services in parallel using multicast
• aggregate the results using the GroupedExchangeAggregationStrategy

The routing for the solution above may look like:

from("direct:serviceFacade")
  .multicast(new GroupedExchangeAggregationStrategy()).parallelProcessing()
    .enrich("http://google.com?q=Foo").enrich("http://google.com?q=Bar")
  .end();

Exchange passed to the direct:serviceFacadeResponse will contain property Exchange.GROUPED_EXCHANGE set to list of results of calls to your services (Google Search in my example).

And that's how could you wire the direct:serviceFacade to Jetty endpoint:

from("jetty:http://0.0.0.0:8080/myapp/myComplexService").enrich("direct:serviceFacade").setBody(property(Exchange.GROUPED_EXCHANGE));

Now all HTTP requests to the service URL exposed by you on ESB using Jetty component will generate responses concatenated from the two calls to the subservices.

Further considerations regarding the dynamic part of messages and endpoints

In many cases using static URL in endpoints is insufficient to achieve what you need. You may also need to prepare payload before passing it to each web service.

Generally speaking - the type of routing used to achieve dynamic endpoints or payloads parameters in highly dependent on the component you use to consume web services (HTTP, CXFRS, Restlet, RSS, etc). Each component varies in the degree and a way in which you can configure it dynamically.

If your endpoints/payloads should be affected dynamically you could also consider the following options:

Preprocess copy of exchange passed to each endpoint using the onPrepareRef option of the Multicast endpoint. You can use it to refer to the custom processor that will modify the payload before passing it to the Multicast's endpoints. This may be good way to compose onPrepareRef with Exchange.HTTP_URI header of HTTP component.

Use Recipient List (which also offers parallelProcessing as the Multicast does) to dynamically create the REST endpoints URLs.

Use Splitter pattern (with parallelProcessing enabled) to split the request into smaller messages dedicated to each service. Once again this option could work pretty well with Exchange.HTTP_URI header of HTTP component. This will work only if both sub-services can be defined using the same endpoint type.

As you can see Camel is pretty flexible and offers you to achieve your goal in many ways. Consider the context of your problem and choose the solution that fits you the best.

If you show me more concrete examples of REST URLs you want to call on each request to the aggregation service I could advice you which solution I will choose and how to implement it. The particularly important is to know which part of the request is dynamic. I also need to know which service consumer you want to use (it will depend on the type of data you will receive from the services).

Answer 2

This looks like a good example where the Content Enricher pattern should be used. Described here

<from uri="direct:performScan"/>
   <enrich uri="ServiceA_Uri_Here" strategyRef="aggregateRequestAndA"/>
   <enrich uri="ServiceA_Uri_Here" strategyRef="aggregateAandB"/>
</route>

The aggregation strategies has to be written in Java (or perhaps some script language, Scala/groovy? - but that I have not tried).

The aggregation strategy just needs to be a bean that implements org.apache.camel.processor.aggregate.AggregationStrategy which in turn requires you to implement one method:

Exchange aggregate(Exchange oldExchange, Exchange newExchange);

So, now it's up to you to merge the request with the response from the enrich service call. You have to do it twice since you have both callA and callB. There are two predefined aggregation strategies that you might or might not find usefull, UseLatestAggregationStrategy and UseOriginalAggregationStrategy. The names are quite self explainatory.

Good luck