Azure NotificationHubClient throws SocketException when used on Azure function intensively

Question

We are using an Azure function to send push notifications via Azure NotificationHubClient. This function is called a lot at peak times on live environment and as a result we got a lot of SocketExceptions thrown from the NotificationHubClient.SendNotificationAsync() method.

Exception details:

Microsoft.Azure.NotificationHubs.Messaging.MessagingException: Unexpected exception encountered TimeStamp:2020-03-27T04:14:35.4655705Z ---> System.Net.Http.HttpRequestException: Cannot assign requested address ---> System.Net.Sockets.SocketException: Cannot assign requested address
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.Azure.NotificationHubs.NotificationHubClient.SendRequestAsync(HttpRequestMessage request, String trackingId, HttpStatusCode[] successfulResponseStatuses, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.NotificationHubs.ExceptionsUtility.HandleUnexpectedException(Exception ex, String trackingId)
   at Microsoft.Azure.NotificationHubs.ExceptionsUtility.TranslateToMessagingException(Exception ex, Int32 timeoutInMilliseconds, String trackingId)
   at Microsoft.Azure.NotificationHubs.NotificationHubClient.SendRequestAsync(HttpRequestMessage request, String trackingId, HttpStatusCode[] successfulResponseStatuses, CancellationToken cancellationToken)
   at Microsoft.Azure.NotificationHubs.NotificationHubClient.SendNotificationImplAsync(Notification notification, String tagExpression, String deviceHandle, CancellationToken cancellationToken)

We create the NotificationHubClient like:

new NotificationHubClient(connectionString, hubName, new NotificationHubClientSettings { OperationTimeout = TimeSpan.FromSeconds(10) });

I've observed that the Microsoft implementation behind is instantiating a new HttpClient on each new NotificationHubClient instantiation - which would probably be the problem(the well known socket exhaustion), but I have no ideea how this could be tackled.

Has anyone confronted with the same problem and managed to fix it some way? Thanks

Answer 1

In the end we managed to overcome the problem by injecting

    IHttpMessageHandlerFactory _httpMessageHandlerFactory;

in our class and pass it to the notificationHubClient in the constructor:

    return new NotificationHubClient(connectionString, hubName, 
    new NotificationHubClientSettings
    {
       OperationTimeout = TimeSpan.FromSeconds(10),
       MessageHandler = _httpMessageHandlerFactory.CreateHandler()
    });