Azure Web Role not starting after upgrade to .Net 4.5 & osFamily 3 (Window 2012)

Question

I have MVC 3 Application that used to work fine on osFamily 1 (Win 2008 SP2) and .Net 4.0 and would like to upgrade to .Net 4.5 and Windows 2012.

My project is upgraded to .Net 4.5 and it builds without any problems and runs in emulator with no problems.

When I deploy to Windows Azure the role i get this message -

"Recycling (Waiting for role to start... Sites were deployed. [2013-06-01T10:43:59Z])"

I remotely logged in to the machine and I found the below error messages in the windows event viewer.

Error Message 1

Log Name:      Application
Source:        Application Error
Date:          6/1/2013 10:15:39 AM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      RD00155D492637
Description:
Faulting application name: WaHostBootstrapper.exe, version: 6.0.6002.18488, time stamp: 0x50a71329
Faulting module name: ntdll.dll, version: 6.2.9200.16420, time stamp: 0x505ab405
Exception code: 0xc0000005
Fault offset: 0x0000000000005d92
Faulting process id: 0xadc
Faulting application start time: 0x01ce5eb0eff6e9f1
Faulting application path: E:\base\x64\WaHostBootstrapper.exe
Faulting module path: D:\Windows\SYSTEM32\ntdll.dll
Report Id: 34c3ae25-caa4-11e2-93ef-00155d492637
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2013-06-01T10:15:39.000000000Z" />
    <EventRecordID>8187</EventRecordID>
    <Channel>Application</Channel>
    <Computer>RD00155D492637</Computer>
    <Security />
  </System>
  <EventData>
    <Data>WaHostBootstrapper.exe</Data>
    <Data>6.0.6002.18488</Data>
    <Data>50a71329</Data>
    <Data>ntdll.dll</Data>
    <Data>6.2.9200.16420</Data>
    <Data>505ab405</Data>
    <Data>c0000005</Data>
    <Data>0000000000005d92</Data>
    <Data>adc</Data>
    <Data>01ce5eb0eff6e9f1</Data>
    <Data>E:\base\x64\WaHostBootstrapper.exe</Data>
    <Data>D:\Windows\SYSTEM32\ntdll.dll</Data>
    <Data>34c3ae25-caa4-11e2-93ef-00155d492637</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event> 

Error Message 2

Log Name:      Application
Source:        .NET Runtime
Date:          6/1/2013 10:15:39 AM
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      RD00155D492637
Description:
Application: CacheDumpGenerator.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Runtime.InteropServices.SEHException
Stack:
   at <Module>.RdGetLocalResource(UInt16*, _RD_RESOURCE**)
   at <Module>.RdGetLocalResource(UInt16*, _RD_RESOURCE**)
   at <Module>._LOCALRESOURCE.GetLocalResourceW(UInt16*, _LOCALRESOURCE**)
   at Microsoft.WindowsAzure.ServiceRuntime.Internal.InteropRoleManager.GetLocalResource(System.String, Microsoft.WindowsAzure.ServiceRuntime.Internal.InteropLocalResource ByRef)
   at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.GetLocalResource(System.String)
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureUtility.GetLocalStoreDirectory()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.InitializeTraceProvider()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.get_TraceProvider()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.WriteLog(System.String, System.Diagnostics.TraceEventType, System.String)
   at Microsoft.ApplicationServer.Caching.CacheDumpGenerator.ProcessDump.Main(System.String[])

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name=".NET Runtime" />
    <EventID Qualifiers="0">1026</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2013-06-01T10:15:39.000000000Z" />
    <EventRecordID>8188</EventRecordID>
    <Channel>Application</Channel>
    <Computer>RD00155D492637</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Application: CacheDumpGenerator.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Runtime.InteropServices.SEHException
Stack:
   at &lt;Module&gt;.RdGetLocalResource(UInt16*, _RD_RESOURCE**)
   at &lt;Module&gt;.RdGetLocalResource(UInt16*, _RD_RESOURCE**)
   at &lt;Module&gt;._LOCALRESOURCE.GetLocalResourceW(UInt16*, _LOCALRESOURCE**)
   at Microsoft.WindowsAzure.ServiceRuntime.Internal.InteropRoleManager.GetLocalResource(System.String, Microsoft.WindowsAzure.ServiceRuntime.Internal.InteropLocalResource ByRef)
   at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.GetLocalResource(System.String)
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureUtility.GetLocalStoreDirectory()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.InitializeTraceProvider()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.get_TraceProvider()
   at Microsoft.ApplicationServer.Caching.AzureCommon.AzureStartupLogger.WriteLog(System.String, System.Diagnostics.TraceEventType, System.String)
   at Microsoft.ApplicationServer.Caching.CacheDumpGenerator.ProcessDump.Main(System.String[])
</Data>
  </EventData>
</Event>

Error Message 3

Log Name:      Application
Source:        Application Error
Date:          6/1/2013 10:15:39 AM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      RD00155D492637
Description:
Faulting application name: CacheDumpGenerator.exe, version: 1.0.4797.0, time stamp: 0x506f41e8
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16451, time stamp: 0x50988aa6
Exception code: 0xc06d007e
Fault offset: 0x000000000003811c
Faulting process id: 0x1934
Faulting application start time: 0x01ce5eb0f71984a5
Faulting application path: E:\plugins\Caching\CacheDumpGenerator.exe
Faulting module path: D:\Windows\system32\KERNELBASE.dll
Report Id: 34e2aca6-caa4-11e2-93ef-00155d492637
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2013-06-01T10:15:39.000000000Z" />
    <EventRecordID>8189</EventRecordID>
    <Channel>Application</Channel>
    <Computer>RD00155D492637</Computer>
    <Security />
  </System>
  <EventData>
    <Data>CacheDumpGenerator.exe</Data>
    <Data>1.0.4797.0</Data>
    <Data>506f41e8</Data>
    <Data>KERNELBASE.dll</Data>
    <Data>6.2.9200.16451</Data>
    <Data>50988aa6</Data>
    <Data>c06d007e</Data>
    <Data>000000000003811c</Data>
    <Data>1934</Data>
    <Data>01ce5eb0f71984a5</Data>
    <Data>E:\plugins\Caching\CacheDumpGenerator.exe</Data>
    <Data>D:\Windows\system32\KERNELBASE.dll</Data>
    <Data>34e2aca6-caa4-11e2-93ef-00155d492637</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>

ServiceConfiguration.Cloud.csfg

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="OpenSpaceAzureDeploymentTesting" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="3" osVersion="*" schemaVersion="2012-10.1.8">
  <Role name="OpenSpaceGUI">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="***" />
      <Setting name="DataConnectionString" value="***" />
      <Setting name="InMaintenanceMode" value="false" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="***" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="RxM6fiusFUxD/10qlvm0xvwMMBy0qbcGpsMRQUUufHD3DlUnJcaASHP0ygRESmvQDPh2ig0FsA0WK65UDrIJxFnBmRTH4WMPmJIlBa5w2Uht/g7wScwZ3UWN6NEQiHpdCPJSayef2iO3VVXRXxZYiHNdP25sMglcEmPQkVVyE4icoVHhFPffjR24uH7uNf8GK83jIbYYAcPKlVOE84pt5a/EPXG1vBQivf9nxRW84Z2jMxmectqi7Ip0tMjWnZx/IE9J28DwhKmnQSn3wlE4/qqGNxVOSp33VvAMDsGCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIwFSSGHiZSSeAGH3U73yNANRQw6W4sIlmu1lpNEXv2mdWzQ==" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2050-01-01T23:59:59.0000000+00:00" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Caching.NamedCaches" value="{&quot;caches&quot;:[{&quot;name&quot;:&quot;default&quot;,&quot;policy&quot;:{&quot;eviction&quot;:{&quot;type&quot;:0},&quot;expiration&quot;:{&quot;defaultTTL&quot;:10,&quot;isExpirable&quot;:true,&quot;type&quot;:2},&quot;serverNotification&quot;:{&quot;isEnabled&quot;:false}},&quot;secondaries&quot;:0},{&quot;name&quot;:&quot;companies&quot;,&quot;policy&quot;:{&quot;eviction&quot;:{&quot;type&quot;:0},&quot;expiration&quot;:{&quot;defaultTTL&quot;:10,&quot;isExpirable&quot;:true,&quot;type&quot;:2},&quot;serverNotification&quot;:{&quot;isEnabled&quot;:false}},&quot;secondaries&quot;:0},{&quot;name&quot;:&quot;person&quot;,&quot;policy&quot;:{&quot;eviction&quot;:{&quot;type&quot;:0},&quot;expiration&quot;:{&quot;defaultTTL&quot;:10,&quot;isExpirable&quot;:true,&quot;type&quot;:2},&quot;serverNotification&quot;:{&quot;isEnabled&quot;:false}},&quot;secondaries&quot;:0}]}" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Caching.DiagnosticLevel" value="1" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Caching.CacheSizePercentage" value="30" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Caching.ConfigStoreConnectionString" value="****" />
      <Setting name="SiteStatus" value="ok" />
    </ConfigurationSettings>
    <Certificates>
      <Certificate name="*.co.uk" thumbprint="**" thumbprintAlgorithm="sha1" />
      <Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="BA399EA1E35B7E3FB00569C0FE1BCA76939EE87F" thumbprintAlgorithm="sha1" />
    </Certificates>
  </Role>
</ServiceConfiguration>

Any help will be appreciated !!

Thanks Bhavesh

---

I did try to remove the Windows.Azure.Caching from my project by removing the package from NuGet, them I tried to deploy to cloud and the web role was deployed successfully but I was unable to browse to my application as it was complaining about in the web.config.

I found the below code was still there in web.config and not cleaned by nuGet.

<dataCacheClients>
    <dataCacheClient>
      <autoDiscover isEnabled="true" identifier="OpenSpaceGUI" />

    </dataCacheClient>
</dataCacheClients>

I removed the above code from web.config and deployed to cloud once again but no luck the web role is recycling in a end-less loop and i found this error details once I remote logged in to the server.

Log Name:      Microsoft-Azure-IISConfigurator/Operational
Source:        Microsoft-Azure-IISConfigurator
Date:          6/5/2013 9:25:38 PM
Event ID:      3303
Task Category: None
Level:         Error
Keywords:      
User:          SYSTEM
Computer:      RD00155D49FC3E
Description:
Exception:System.Runtime.InteropServices.COMException (0xC00CEF03): Exception from HRESULT: 0xC00CEF03

Server stack trace: 
   at Microsoft.Web.Administration.Interop.AppHostWritableAdminManager.CommitChanges()
   at Microsoft.Web.Administration.Configuration.CommitChanges()
   at Microsoft.Web.Administration.ConfigurationManager.CommitChanges()
   at Microsoft.Web.Administration.ServerManager.CommitChanges()
   at Microsoft.WindowsAzure.ServiceRuntime.IISConfigurator.WasManager.DeploySite(String roleId, WASite roleSite, String appPoolName, String sitePath, String iisLogsRootFolder, String failedRequestLogsRootFolder, List`1 bindings, List`1 protocols, FileManager fileManager, WAAppPool defaultAppPoolSettings, String roleGuid, String& appPoolSid, List`1 appPoolsAdded, String configPath)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.WindowsAzure.ServiceRuntime.IISConfigurator.WasContainer.DeploySite(String roleId, WASite roleSite, String appPoolName, String sitePath, String iisLogsRootFolder, String failedRequestLogsRootFolder, List`1 bindings, List`1 protocols, FileManager fileManager, WAAppPool defaultAppPoolSettings, String roleGuid, String& appPoolSid, List`1 appPoolsAdded, String configPath)
   at Microsoft.WindowsAzure.ServiceRuntime.IISConfigurator.IISConfigurator.Deploy(String roleId, WebAppModel webAppModel, String roleRootDirectory, String sitesDestinationRootDirectory, String diagnosticsRootDirectory, String roleGuid, String configPath)

I have no clue now why this is not working, any help will be appreciated.

Thanks

Answer 1

You need to run your application in OS family 3 if you want .NET 4.5 support. You need to specify this in your ServiceConfiguration.Cloud.cscfg file

<ServiceConfiguration serviceName="XXX" 
    xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration"
    osFamily="3" 
    osVersion="*" 
    schemaVersion="2012-10.1.8">

It may not be possible to upgrade your instance and you may have to deploy a new one.

More details on the Azure blog

---

Update

WaHostBootstrapper.exe will fail when there is something wrong with your azure service (most often without any helpful details) so it is hard to determine exactly what is wrong. It could be something as simple as missing references.

From what you have posted it looks like some of your Microsoft.WindowsAzure.Plugins are having a hard time as well. I would try to remove them and see if the role still fails.

Side note - it is no longer necessary to specify Remote Access in your cscfg file. From ScottGu's blog :

Windows Azure Cloud Services support the ability for developers to RDP into web and
worker role instances. This can be useful when debugging issues.

Prior to today’s release, developers had to explicitly enable RDP support during development – prior to deploying the Cloud Service to production. If you forgot to enable this, and then ran into an issue in production, you couldn’t RDP into it without doing an app update and redeploy (and then waiting to hit the issue again).

With today’s release we have added support to enable administrators to dynamically
configure remote desktop support – even when it was not enabled during the initial
app deployment. This ensures you can always debug issues in production and never
have to redeploy an app in order to RDP into it.