Rebus crash in .NET async core

Question

I am trying to upgrade an existing website from Rebus 0.45 to Rebus 2.0 and have run into a problem where System.Web.ThreadContext.AssociateWithCurrentThread crashes with a null reference. I have absolutely no idea about why, so I am looking for possible causes that I can look for.

The only trace I have is the Windows Event Log that registers the following when trying to send a message:

An unhandled exception occurred and the process was terminated.

Application ID: /LM/W3SVC/1/ROOT/website

Process ID: 14460

Exception: System.NullReferenceException

Message: Object reference not set to an instance of an object.

StackTrace:    at System.Web.ThreadContext.AssociateWithCurrentThread(Boolean setImpersonationContext)
   at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallbackPossiblyUnderLock(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallback(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.Post(SendOrPostCallback callback, Object state)
   at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.PostAction(Object state)
   at System.Threading.Tasks.AwaitTaskContinuation.RunCallback(ContextCallback callback, Object state, Task& currentTask)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.AwaitTaskContinuation.<>c.<ThrowAsyncIfNecessary>b__18_0(Object s)
   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

The website is running ASP.NET MVC 3.0 with a Castle Windsor dependency injector and uses Rebus.SqlServer for transport setup as oneway client:

var busConfigurer = Configure.With(new CastleWindsorContainerAdapter(container))
  .Logging(l => l.Use(new RebusLoggerFactory()))
  .Routing(t => t.TypeBased().AddEndpointMappingsFromAppConfig())
  .Subscriptions(s => s.StoreInSqlServer(dbConnectionString, "RebusSubscriptions_v2"))
  .Transport(t => t.UseSqlServerAsOneWayClient(dbConnectionString, "RebusMessages_v2"))
  Start();

IIS is setup for ".NET 4.5" application pool.

The web site starts and the bus logs:

- Supplied connection string will be modified to enable MARS
- Database already contains a table named 'RebusMessages_v2' - will not create anything
- Supplied connection string will be modified to enable MARS
- Starting periodic task 'CleanupTrackedErrors' with interval 00:01:00
- Starting bus 1
- Started

At a later time a message is sent via Rebus that logs:

- Sending SubmissionCreatedEvent -> SelfService.Service.input

... and boom! the whole application crashes.

Before sending the message the application has interacted with NHibernate to store details about the operation.

Update

By changing Bus.Send(msg) to Bus.Send(msg).Start() I can trigger the same error synchronously and get the following stack trace when debugging in Visual Studio:

Exception thrown: 'System.NullReferenceException' in System.Web.dll
Exception thrown: 'System.NullReferenceException' in mscorlib.dll
System.Transactions Critical: 0 : 
<TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Critical"><TraceIdentifier>http://msdn.microsoft.com/TraceCodes/System/ActivityTracing/2004/07/Reliability/Exception/Unhandled</TraceIdentifier><Description>Unhandled exception</Description><AppDomain>/LM/W3SVC/1/ROOT/website-2-131225642632547395</AppDomain>
<Exception><ExceptionType>System.NullReferenceException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType><Message>Object reference not set to an instance of an object.</Message><StackTrace>   at System.Web.ThreadContext.AssociateWithCurrentThread(Boolean setImpersonationContext)
   at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallbackPossiblyUnderLock(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallback(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.Post(SendOrPostCallback callback, Object state)
   at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.PostAction(Object state)
   at System.Threading.Tasks.AwaitTaskContinuation.RunCallback(ContextCallback callback, Object state, Task&amp;amp; currentTask)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.AwaitTaskContinuation.&amp;lt;&amp;gt;c.&amp;lt;ThrowAsyncIfNecessary&amp;gt;b__18_0(Object s)
   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()</StackTrace><ExceptionString>System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Web.ThreadContext.AssociateWithCurrentThread(Boolean setImpersonationContext)
   at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallbackPossiblyUnderLock(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.CallCallback(SendOrPostCallback callback, Object state)
   at System.Web.LegacyAspNetSynchronizationContext.Post(SendOrPostCallback callback, Object state)
   at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.PostAction(Object state)
   at System.Threading.Tasks.AwaitTaskContinuation.RunCallback(ContextCallback callback, Object state, Task&amp;amp; currentTask)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.AwaitTaskContinuation.&amp;lt;&amp;gt;c.&amp;lt;ThrowAsyncIfNecessary&amp;gt;b__18_0(Object s)
   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()</ExceptionString></Exception></TraceRecord>

Update II

Replacing Bus.Publish(msg) with bus.Advanced.Routing.Send("SelfService.Service.input", msg) works. So apparently there is something wrong in the subscription/routing stuff as Send(...) bypasses the that.

Update III

Apparently adding <httpRuntime targetFramework="4.6" ... in the web.config solved the problem (so far, it still seems a bit erratic). The web.config already had <compilation ... targetFramework="4.6">:

  <system.web>
    <httpRuntime targetFramework="4.6" maxRequestLength="5000" requestValidationMode="2.0" />
    <compilation debug="true" targetFramework="4.6">
      <assemblies>...</assemblies>
    </compilation>
    <pages validateRequest="false">
      <namespaces>...</namespaces>
    </pages>
  </system.web>

Answer 1

The fix turned out to be to make sure the website is running .NET 4.5 or higher:

<system.web>
  <httpRuntime targetFramework="4.6" />
  <compilation targetFramework="4.6">
    ...
  </compilation>
</system.web>

Apparently the ASP.NET team made ASP.NET more task aware in .NET 4.5.