Publishing an ASP.NET MVC2 site with Web Deploy

Question

I currently use Web Deploy, http://learn.iis.net/page.aspx/346/web-deploy/ to publish my MVC2 app. It used to work well, but now it is got to the point where I can't continue using it:

When the MVC app was small and had only a few users it was easy to publish. Just right click the project in Visual Studio and choose "Publish". And because there were only a few users it was easy to find a time when no one was using the site to do a quick update.

Then the app got bigger and had a few more users. The "Publish" action started taking longer and longer and occasionally timing out. Even when I recycled the app pool before deploy it still took a long time.

Also it became harder to find a time when no one was using the site so the update could be done without affecting anyone.

Then the "Publish" action started timing out every single time, and I had to switch to manual deployment as per this earlier unanswered question: Visual Studio 2010 - web deploy times out - what to do?

Now the manual deploy is taking longer and longer, from 5 to 20 minutes. And the number of users has grown significantly, so the deployment always affects someone (slow response times, timeouts, site unavailable, etc)

So what can I do? Is there a better alternative to using web deploy?

Edit:

Today's deployment took 18 minutes to publish just 49 changed files. The situation is just ridiculous and is one of the biggest weaknesses of our site right now. So I'm starting a decent sized bounty in the hopes of solving this.

Some more questions that may lead to a solution:

• Why would it take so long when only a few files have been changed?
• Why does the web deploy zip always include the entire codebase and not just changed files?
• Why don't I just manually copy the changed files myself and skip the whole web deploy? But it is hard to manually work out what files have changed. I use SVN - does it have a way to output only files that have changed between two branches?
• What other questions should I be asking but haven't thought of yet?

In reply to answers:

Re: http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity_24.html This is exactly how I was doing the deploy, and would be an ideal method. Web deploy does correctly identify which files have changed, however it times out and no publish occurs. There are around 2500 files in the solution, perhaps it is taking too long to identify which ones are changed? Or it could be that publish has a short timeout value and that just uploading the 15mb zip file uses all that time up.

I do have full control over the server, and it does support web deploy. There are actually 2 servers: the primary live server, and a redundant server that we keep ready in case the first falls over. So any solution has to be easy to deploy to more than one server (web deploy was ideal until it stopped working).

The suggestion of creating a new folder for each release and then just changing IIS to point to that new folder sounds like it will result in lower downtime/slowtime when during the publish. But it is a very manual process and I would prefer something more automated.

Edit #2

I have managed to narrow it down, and found exactly where it is slow - but not why. This is from the deploy log:

[9/02/2011 12:11:56 a.m.] Performing synchronization pass #1.
[9/02/2011 12:11:56 a.m.] Parameter entry 'IIS Web Application Name/1' is applicable to 'iisApp/C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp' because of its scope.
[9/02/2011 12:11:56 a.m.] Parameter entry 'IIS Web Application Name/2' is applicable to 'setAcl/C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp' because of its scope.
[9/02/2011 12:11:56 a.m.] Parameter entry 'IIS Web Application Name/2' is applicable to 'setAcl/C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp' because of its scope.
[9/02/2011 12:11:56 a.m.] Parameter entry 'Add write permission to App_Data Folder/1' is applicable to 'setAcl/C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp\App_Data' because of its scope.
[9/02/2011 12:11:56 a.m.] Source createApp (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp) does not match destination (Default Web Site/virtual-dir/) differing in attributes (isDest['False','True']). Update pending.
[9/02/2011 12:11:56 a.m.] Update operation on createApp (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp) skipped because of rule CreateApplicationRule.
[9/02/2011 12:11:56 a.m.] Source filePath (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp\App_Data\Create.sql) does not match destination (Default Web Site/virtual-dir/App_Data\Create.sql) differing in attributes (size['259691','259697'],lastWriteTime['02/08/2011 10:45:20','02/06/2011 03:48:16']). Update pending.

[400 lines of file updates skipped, time expired 2 seconds ....]

[9/02/2011 12:11:58 a.m.] Delete operation on filePath (Default Web Site/v2/zzz_app_offline.htm) skipped because of rule DoNotDeleteRule.
[9/02/2011 12:11:58 a.m.] Source setAcl (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp) does not match destination (Default Web Site/virtual-dir/) differing in attributes (isDest['False','True'],setAclUser,setAclAccess). Update pending.
[9/02/2011 12:11:58 a.m.] Updating setAcl (Default Web Site/virtual-dir/).
[9/02/2011 12:13:47 a.m.] Source setAcl (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp) does not match destination (Default Web Site/virtual-dir/) differing in attributes (isDest['False','True'],setAclUser,setAclAccess). Update pending.
[9/02/2011 12:13:47 a.m.] Updating setAcl (Default Web Site/virtual-dir/).
[9/02/2011 12:17:11 a.m.] Source setAcl (C:\src\Site.2010\Site.UI\obj\Release\Package\PackageTmp\App_Data) does not match destination (Default Web Site/virtual-dir//App_Data) differing in attributes (isDest['False','True'],setAclUser,setAclAccess). Update pending.
[9/02/2011 12:17:11 a.m.] Updating setAcl (Default Web Site/virtual-dir//App_Data).
[9/02/2011 12:17:11 a.m.] The dependency check 'DependencyCheckInUse' found no issues.
[9/02/2011 12:17:11 a.m.] The synchronization completed in 1 pass(es).

The cause of the slowness is the "Updating setAcl" component. I am examining the ACLs of the development box and server box to see what is different. However it seems like an extremely bad idea to copy the ACL from a dev box to a server box! I already had the ACL set up just fine on the server.

Answer 1

@JK from the info you provided it feels like a timeout issue. I agree with @TroyHunt 2500 files in 15 megs should delploy quickly. In particular the the output showing a delay while applying ACLs (whether or not they need to be changed). If it were me, I would start out doing some non-web deploy health checks. A few thoughts come to mind

• are the webservers in a domain or workgroup?
• what does dcdiag show ?
• what does netdiag show ?
• are the dev boxes and the prod boxes in the same domain?

Is there a chance you are trying to apply users or groups that don't exist any more, are disabled , or come from domains other than the web server domain? It's possible your organization has a domain hierarchy with child domains or domain trusts, that are valid, but communication is blocked in the production data center.

I think i would also manually look at the ACLs and see if there are entries that can't be resolved (they show up as SIDS before resolution).

HTH, -eric