BizTalk: how to limit number of connections to a wcf service?

Question

I developed a BizTalk application that receives as input a file that contains a bunch of messages. I use the BizTalk XML disassembler component to 'debatch' the file in sepereate messages. Each of those messages is picked up from the MessageBox by an orchestration that transforms the message and calls a wcf service.

The problem I am experiencing now is that each batch contains 1000 messages, and that those 1000 messages all seem to call the wcf service at once. The wcf service is "bombed" by those messages and is configured to process only 10 messages in parallel (each call has to process data and put data in a database) and returns a bunch of "Too Busy" exceptions back to BizTalk. I configured the wcf adapter to retry the connection again after 1 minute.

The end result is that BizTalk first debatches the messages, then bombs the wcf service with all 1000 messages, gets a bunch of "Too busy" exceptions, then waits while doing nothing, until 1 minute is passed, then bombs it again, and so on.

The processing would be much more efficient if I could configure BizTalk to open max 10 connections to that specific wcf service, but as far as I know, this is not possible. (The wcf service is configured to use net.tcp.)

I did already try the throttling settings of the host in several different ways, but either it is not helping, or it is making the application unbearable slow. Also the throttling in BizTalk seems to be implemented in a way that it first bombs a service, then notices that it was bombing, then waits a while doing nothing, and then lifts the throttle and start bombing again. It seems much better to trickle the requests/messages, so that they are much more evenly spread in time. I would like to configure the WCF adapter to receive max 4 messages per second for example. The throttling that is possible now says something like: over the sliding window of 5 seconds, i want throttling to be activated if there are more than 20 messages. But that is not the same, because it allows a "burst" effect.

Any ideas how i can improve throughput?

Answer 1

Use the BizTalk singleton pattern. This is ugly. But BizTalk elegant architecture create ugliness when it meets the real world.