Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Periodical ServiceBus error "The X.509 certificate CN=servicebus.windows.net is not in the trusted people store" when using Azure WebSites

I have several websites running in Azure, intensively using ServiceBus (also hosted in Azure), all in one region.

Sometimes (once every 2-3 days) I have same error in all web sites at the same time (during reading/waiting for messages):

Microsoft.ServiceBus.Messaging.MessagingCommunicationException: 
 The X.509 certificate CN=servicebus.windows.net is not in the trusted people store.
  The X.509 certificate CN=servicebus.windows.net chain building failed. 
   The certificate that was used has a trust chain that cannot be verified. 
    Replace the certificate or change the certificateValidationMode. 
     A certificate chain could not be built to a trusted root authority.

Full stacktrace:

Microsoft.ServiceBus.Messaging.MessagingCommunicationException: The X.509 certificate CN=servicebus.windows.net is not in the trusted people store. The X.509 certificate CN=servicebus.windows.net chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A certificate chain could not be built to a trusted root authority.
 ---> System.ServiceModel.Security.SecurityNegotiationException: The X.509 certificate CN=servicebus.windows.net is not in the trusted people store. The X.509 certificate CN=servicebus.windows.net chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A certificate chain could not be built to a trusted root authority.
 ---> System.IdentityModel.Tokens.SecurityTokenValidationException: The X.509 certificate CN=servicebus.windows.net is not in the trusted people store. The X.509 certificate CN=servicebus.windows.net chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A certificate chain could not be built to a trusted root authority.

   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.ServiceModel.Channels.SslStreamSecurityUpgradeInitiator.InitiateUpgradeAsyncResult.OnCompleteAuthenticateAsClient(IAsyncResult result)
   --- End of inner exception stack trace ---

Server stack trace: 
   at Microsoft.ServiceBus.Messaging.Channels.SharedChannel`1.CreateChannelAsyncResult.<GetAsyncSteps>d__7.MoveNext()
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.EnumerateSteps(CurrentThreadType state)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
   at Microsoft.ServiceBus.Common.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)

Exception rethrown at [0]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.Channels.SharedChannel`1.OnEndCreateInstance(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.SingletonManager`1.EndGetInstance(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__2(RequestAsyncResult thisPtr, IAsyncResult r)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [1]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory`1.RedirectContainerSessionChannel.RequestAsyncResult.<>c__DisplayClass17.<GetAsyncSteps>b__a(RequestAsyncResult thisPtr, IAsyncResult r)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [2]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory`1.RedirectContainerSessionChannel.EndRequest(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__4(RequestAsyncResult thisPtr, IAsyncResult r)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [3]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.CloseOrAbortLinkAsyncResult.<GetAsyncSteps>b__7(CloseOrAbortLinkAsyncResult thisPtr, IAsyncResult a)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [4]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.AbandonPrefetchedMessagesCloseAbortAsyncResult.<GetAsyncSteps>b__41(AbandonPrefetchedMessagesCloseAbortAsyncResult thisPtr, IAsyncResult r)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [5]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndClose(IAsyncResult result)
   --- End of inner exception stack trace ---
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndClose(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.ClientEntity.OnClose(TimeSpan timeout)
   at Microsoft.ServiceBus.Messaging.ClientEntity.Close(TimeSpan timeout)
   at Olekstra.Common.QueueReader.<>c__DisplayClass3.<StartTask>b__2(Boolean force)

I'm using up-to-date version of Microsoft.ServiceBus.dll NuGet package (2.4.0.0), so answers from Google "certificate validation is off by default in ServiceBus 1.8" are useless - I had not turned any validation to ON, and moreover - it works many hours in a row and fails only once every 2-3 days.

Similar answer about self-hosted applications are not suitable too - website is running inside Azure, hosting VMs are manages by MS staff, I'm not allowed to update any root certificates.

Does anybody know why ServiceBus client decides to check SSL cert sometimes and how to disable this behaviour?

UPD:

I added <add key="Microsoft.ServiceBus.X509RevocationMode" value="NoCheck"/> in appSettings in web.config two weeks ago - no difference.

Also Reflector shows that 'NoCheck' value is default in Microsoft.ServiceBus.Configuration.ConfigurationHelpers.GetCertificateRevocationMode()

like image 914
Dmitry Avatar asked Sep 09 '14 08:09

Dmitry


1 Answers

Another approach that will solve this problem but has security implications: https://stackoverflow.com/a/39415887/5869

like image 65
George Tsiokos Avatar answered Sep 21 '22 07:09

George Tsiokos