Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What Azure .NET SDK EventHubClient instance methods are threadsafe?

I'm writing code that will be publishing messages from multiple threads to an Azure Event Hub in C# using the EventHubClient. The documentation for EventHubClient contains the fairly standard boiler plate.

"Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe."

There is no additional documentation as to thread safety in any of the four send methods I would most expect to be thread safe. Were I to believe that the send methods are not threadsafe then I would end up creating a new EventHubClient instance each time I wished to send to a message. Since the underlying tcp connection is apparently reused unless steps are taken this may not have too much overhead. Similar issues arise with partitioned senders though given that there is an async method to create one, they might well have their own AMQP connection.

Are some, if not all, instance methods of EventHubClient thread safe despite the documentation?

And for any Azure folks would it be possible to have this clarified in the documentation? This sort of documentation issue (assuming it is wrong as seems likely) appears to affect Azure Table as well and is generally common within the MSDN docs. With regards to EventHub this is in contrast to the clear thread safety statement of Kafka and AWS Kinesis at least does not explicitly label everything as unsafe. I did not find EventHubs in the open source portion of the SDK so could not check myself.

like image 780
cacsar Avatar asked Mar 18 '23 21:03

cacsar


1 Answers

TLDR:

  1. All critical runtime operations (aka data-plane) in the .NET SDK are thread-safe.
  2. Create EventHubClient object once and re-use

The Story

ServiceBus SDK exposes two patterns to create senders:

  1. Basic
  2. Advanced

For Basic version - developer will directly use EventHubClient.CreateFromConnectionString() API and doesn't worry about managing MessagingFactory objects (connection gu's). SDK will handle reusing the MessagingFactory across all EventHubClient instances as long as the connection string is same - a literal match of all keys and values - is done in the SDK for this reuse.

For an Advanced developer who will need a bit more control at connection level, SB SDK provides MessagingFactory.CreateFromConnectionString() and from this developer can create the EventHubClient instance.

All instance methods of EventHubClient - to send to EventHubs are strictly thread-safe. In general, all data-plane operations are... However, while reading from EventHubs, API is optimized for, this pattern. while(true) { var events = eventHubPartitionReceiver.receive(100); processMyEvents(events); } So, for ex: properties like, EventHubReceiver.RuntimeInformation - is populated after every receive call without any synchronization. So, even though the actual receive API is thread-safe - the subsequent call to RuntimeInformation isn't - as it is rare for anyone to park multiple receive calls on an instance of PartitionReceiver.

Creating a new instance of EventHubClient in each component to start send messages is the default pattern - and the ServiceBus SDK will take care of reusing the underlying MessagingFactory - which reuses the same physical socket (if the connection string is same).

If you are looking for real high throughput scenarios then you should design a strategy to create multiple MessagingFactory objects and then Create an EventHubClient each. However - make sure that you have already increased the Thruput units for your EventHub on the Portal before trying this as the default is just 1 MBPS - cumulative of all 16 partitions.

Also, if the Send pattern you are using is Partitioned Senders - they all will also use the same underlying MessagingFactory - if you create all Senders from the same eventHubClient(.CreatePartitionedSender()) instance.

like image 51
Sreeram Garlapati Avatar answered May 02 '23 15:05

Sreeram Garlapati