EventHub Exception :Cannot allocate more handles to the current session or connection

Question

I have C# console application through which I am sending data to event hub frequently.This console application basically read some data from SQL storage and start pushing data to event hub.

This entire program run in endless loop/fashion like in while control whenever it receive any data in SQL it pulls data from there and start sending to Event hub.

But at some moment I got this error.

"Cannot allocate more handles to the current session or connection.The maximum number of handles allowed is 4999.Please free up resources any try again. at Microsoft.ServiceBus.Common......

When I restarted this console application its working fine.But I don't know why I got this error .

Please help me.

Thanks & Regards,

RK

Answer 1

TLDR: In the 'send to EventHub logic', make sure that you are reusing (cache'ing) the same sender instance.

The Why:

EventHubs is designed to support very large scale high-thruput low-latency event'ing systems. Hence, we chose to rely on a very performant protocol for all Runtime Operations - namely Amqp. The goodness of Amqp Protocol comes into play when the application built on top of it fully leverages its strengths. This is how EventHubs Object Model maps to Amqp artifacts:

1. EventHubClient maps to one single AmqpConnection. Cardinality is 1:1. If exact same ConnectionString is specified while Creating EventHubClient.CreateFromConnectionString - The underlying physical socket will be shared - but the amqp Artifact - AmqpConnection is still different.
2. Whenever client invokes EventHubClient.Send(EventData), internally, EventHubClient creates 1 AmqpSession and 1 AmqpLink in that Session on the AmqpConnection created by EventHubClient. This Session and Link are re-used as long as the same EventHubClient instance is used for subsequent Sends.
3. Whenever any Management operation is performed on the EventHubClient - since, mgmt. operations (like getPartitionInfo) are always request-response and require 2-way communication - EventHubClient creates 1 AmqpSession and 2 AmqpLink's in that Session - one Link for the Request & other link for Response (Ex: imagine the result of a REST Get call).
4. Whenever, any child entities are Created from EventHubClient - like EventHubSender or EventHubReceiver - EventHubClient creates a brand new AmqpSession && an AmqpLink in that Session.

Key takeaways for the Client Application using eventhub SDK are:

Every Time an EventHubClient instance is created - a real physical socket is created underneath.

Every Time an EventHubSender or an EventHubReceiver instance is created that socket created by EventHubClient is re-used and on top of it: (a) an AmqpSession is created - no. of AmqpSessions are limited to 5k per connection - I guess this is the limit your client application is hitting above. (b) AmqpLink is created inside that Session - which will inturn trigger Entity Resolution in EventHubs Service (which is a tiny bit expensive compared to sending on an existing EventHubSender).

More on Event Hubs.