Is smooth deployment possible with componentized ASP.NET MVC apps?

Question

That's probably not MVC specific, it might as well applicable to ASP.NET WebForms, but we've been experiencing it on MVC2 so far.

Whenever we start remote deployment with MSDeploy we get a brief (5-6 seconds of) "server error" page for our requests until new deployment takes place. Here is the error text:

Server Error in '/' Application.

Could not load file or assembly 'Some.Assembly' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.IO.FileLoadException: Could not load file or assembly 'Some.Assembly' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.1

Here are the stack traces shown in the error page:

[FileLoadException: Could not load file or assembly 'Some.Assembly' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)]
   System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) +0
   System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) +39
   System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection, Boolean suppressSecurityChecks) +132
   System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) +144
   System.Reflection.Assembly.Load(String assemblyString) +28
   System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective) +46

[ConfigurationErrorsException: Could not load file or assembly 'Some.Assembly' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)]
   System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective) +618
   System.Web.Configuration.CompilationSection.LoadAllAssembliesFromAppDomainBinDirectory() +209
   System.Web.Configuration.CompilationSection.LoadAssembly(AssemblyInfo ai) +130
   System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig) +178
   System.Web.Compilation.BuildManager.GetPreStartInitMethodsFromReferencedAssemblies() +94
   System.Web.Compilation.BuildManager.CallPreStartInitMethods() +332
   System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException) +591

[HttpException (0x80004005): Could not load file or assembly 'Some.Assembly' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +8950644
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +97
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +256

The assembly which is causing that failure (Some.Assembly) is not the actual web assembly but one of other components under it's "References", signed with a .snk.

After 5-6 seconds, the site goes up and the error goes away.

That's certainly not the desired behavior. I wonder if we are doing something wrong in terms of wiring up the components together. Should there be another approach to ensure a smooth deployment? Or could this be a bug with MVC2 itself?

P.S. IIS overlapped rotation is enabled, it's the default anyways.

Here are the components:

• A.dll
• Some.Assembly.dll (<--- this is the failing one, depends on A.dll)
• B.dll (depends on A.dll and Some.Assembly.dll)
• Web.dll (this is the web application, depends on all of the above)

All the dependencies are established using regular assembly references with Copy Local enabled. All dll's are part of the solution as separate projects.

MSDeploy only deploys binaries, not the source.

Answer 1

Overlapping rotation only applies to pool recycling and is not intended to facilitate the type of deployment you're describing or require. When overlapping rotation is enabled, existing requests in a pool's "outgoing" worker process will be allowed to complete whilst new requests will be sent to the new worker process created. It's a mechanism to gracefully handover to a new worker process without pulling the rug from underneath existing requests. That's all it does.

Shadow copy folders are used for this purpose:

What is the “Temporary ASP.NET Files” folder for? (My Answer)

Again they are not intended to provide a mechanism to facilitate keeping a whole existing code base running whilst you upload a new site.

When you deploy an ASP.NET application the site will misbehave. Whilst you are copying your assemblies these files will be locked (probably exclusively) by whatever process is handling the upload (WebDAV or FTP).

This causes the exceptions you've observed and are most likely because the shadow copy mechanism can't read the new assemblies until they are written (and WebDAV or FTP removes the write locks).

Additionally, any pages that have dependencies on these assemblies may not (shadow) compile if expected method signatures have changed or been removed until the correct pages have been uploaded. Or if the pages/views upload first that depend on functionality in assemblies that haven't yet been deployed you will also get errors.

The whole site will be in an inconsistent state until the last file is deployed.

There is no built-in mechanism in IIS to ensure an "atomic" deployment, i.e. load all your stuff then switch over to running that. IIS will keep serving requests to the site and ASP.NET will still keep detecting file changes as you upload the application.

The only way to deploy an application without generating these errors this is to enable a special page called App_Offline.htm before deployment and then rename or remove after deployment:

App_Offline.htm - Scott Guthrie
App_Offline.htm and working around the "IE Friendly Errors" feature

You may also find this article useful:

How to: Prepare to Deploy a Web Project

There is a slightly more convoluted alternative which involves having two folders, for example:

d:\websites\site\www-A d:\websites\site\www-B

The running site is pointing at d:\websites\site\www-A, meanwhile you deploy into d:\websites\site\www-B. When you're ready you switch the site to the d:\websites\site\www-B folder.

When you come to deploy the next built you deploy into d:\websites\site\www-A and switch to that when you're happy.

The drawback is that you need to be on your toes and remember which folder is which.

Also any user uploaded content would need to be synchronised between the two folders (although you could have a third folder mapped into to a virtual directory for stuff like that).

Answer 2

My deployment script stops the website and displays a "Please wait while the site is being updated..." page, which refreshes every 15 seconds. Since the deployment takes under a minute it seems to fix the problem.