Automating deployment of .Net Web App to Azure & SQL Azure & Azure Blobs

Question

Nutshell :

I'd like some help / info / resources r.e. setting up a Team Build, MsBuild, TFS 2010 automated deployment for my Web App to Azure (inc. all the DB bits).

Ideally I'd like to have a process that I can fire off from the VS2010 Team Explorer UI "Queue New Build", then just keep an eye on its progress, releasing me to work on something else. Options to delve into logging for any issues, and know that my process is robust, complete and totally non-manual, i.e. :

• Backs up all my Live data (SQL Azure and Azure Blobs)
• Deploys any DB schema changes (contained in my DB project)
• Deploys any data changes to my core data (e.g. config data etc - which I have in my Post-Deployment scripts)
• Does things sensibly (e.g. using compression for deployment packages to save time & bandwidth)
• Cover my silly backside (e.g. seamlessly rolling back failed changes)
• Keeps app 100% running during deployments (failed or successful) e.g. sessions left intact, minimal chances of data loss
• Keep detailed logs of processes progress at each stage for fixing any issues
• Keep everything that should be Source Controlled well... source controlled

Background / Dream / Goal :

At my last FT job we had a pretty sweet automated deployment setup for our hosted Web Apps, using CC.Net (to manage the process), CCTray & the CC.NET web UI to monitor and control, Code Generation (CodeSmith + NetTiers templates for data access & entities), MSBuild, VS Databse Projects, a few .bat scripts, and some handy utilities like PsExec etc to help out with little bits and pieces. I didn't set it up, but have some experience managing it, dealing with issues etc.

It was (98% of the time) a lovely experience to deploy. You'd make sure TFS was up to date, double click CCtray, right click on a project and then click "Force Build", sit back and watch Green => Yellow => Green.

Great !!

Current Situation :

I run my own Micro ISV, and my main project is an App on Azure (in Beta). I'd very much like to replicate the kind of deployment experience I had before - I'm even considering moving out of Azure to dedicated servers - just because I know I can setup an automated deployment system there.

My main stopping point is the DB bits, seems like a nightmare. Maybe I'm missing some great free tool or library which would do the job, I really hope so, but I also could really do with someone experienced with this to point me in the right direction for a "Best Practice" solution to wrap up all the little bits neatly.

I have scoured Google, read and read, burnt hours and hours, but what I seem to find atm is half solutions, not quite right for my project and needs, based on expensive tools I can't afford (near $0 budget), or is plain over my head and a bit incomprehensible & scary.

Now I'm NO Sys Admin, but with enough time I can generally work out what I need to do for these sort of things.

However, I don't have ANY time right now, and the success of my whole project really depends on me being able to cut out the horrendous 40min+ manual deployment wastage I currently endure.

I want to be able to get some user feedback, find a bug, or code an improvement and confidently just fire off a deploy and crack on with something else.

The extra issues with development for Azure in its current state (as opposed to dedicated servers), and the currently fairly poor tooling support from MS (I know there's lots of improvements coming, but I need something right now) - has left me swimming in a sea of "I don't know"'s & "I'm not sure"'s and tends to end in one big :

"I give up + a manual deploy for almost an hour + a little sobbing inside as my dream of deployment heaven dies once again" :(

But I know people out there more proficient, knowledgeable and experienced than myself have cracked this one for themselves. I just can’t seem to find the info I am sure is out there.

So if anyone has some good resources, tips, links, comments, or opinions on this, I'd love to hear.

Details Of My Setup :

App up and running in Azure (which is in Beta - partly due to not having the auto deploy setup), running in a Production slot, I haven't bothered with a staging slot, as some issues with subdomains / DNS / the auto generated Url has made that look painful / not feasible.

Azure / App :

App is

1 Web Role

• ASP.NET 4
• MVC 2
• EF 4
• SQL Azure
• Azure Blob Storage

1 Worker Role - this runs some scheduled tasks, and works with same DB and Blob Storage

• SQL Azure
• Azure Blob Storage

The 2 Roles communicate with the Azure queue system (or will do shortly)

Locally :

Datacenter 08 (DC) + Hyper V - VM for TFS 2010 - VM for a Linux firewall

Dev Box 1 (Win 7) - VS 2010 / VS 08 - SQL 08 R2 / 05

Dev Laptop 2 - as above.

I tend to run these together all the time (so I never need to stop to wait for anything) with the FANTASTIC free tool Synergy to bind Keyboard, Mouse, Clipboard together.

Some Of The Stuff I've Read :

I have read what I have found and some of its great stuff, so I am also posting these links here to help other's struggling with this stuff, but none of it quite seems to do the trick, or maybe I don't get the trick maybe I'm missing something ?

http://deploytoazure.codeplex.com/

How do I manage and publish a database with my MVC2 application on Azure?

How can I automate the "generate scripts" task in SQL Server Management Studio 2008?

http://www.koltovich.com/blog/DeployingAzureProjectFromTFS2010BuildServer.aspx

http://msdn.microsoft.com/en-us/library/ff803365.aspx

http://msdn.microsoft.com/en-us/library/gg432988.aspx

http://www.jimzimmerman.com/blog/2010/03/16/Deploying+An+Azure+Project+Using+TFS+2010.aspx

http://archive.msdn.microsoft.com/azurecmdlets

http://selfpacedazure.web.officelive.com/Documents/Windows%20Azure%20Platform%20Articles%20from%20the%20Trenches.pdf

http://msdn.microsoft.com/en-us/library/gg651132.aspx

http://social.technet.microsoft.com/wiki/contents/articles/overview-of-tools-to-use-with-sql-azure.aspx

http://msdn.microsoft.com/en-us/library/ms178078.aspx

http://blog.syntaxc4.net/post/2011/05/13/Continuous-Integration-in-the-Cloud.aspx

http://blog.syntaxc4.net/post/2009/12/31/Synchronizing-a-Local-Database-with-the-Cloud-using-SQL-Azure-Sync-Framework.aspx

http://social.technet.microsoft.com/wiki/contents/articles/overview-of-tools-to-use-with-sql-azure.aspx

http://social.technet.microsoft.com/wiki/contents/articles/developing-and-deploying-with-sql-azure.aspx

http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx

http://www.scarydba.com/2011/04/25/sql-azure-deployments/

Disclaimer / Forum Abuse Minimisation Blurb :

Like I say I am NO Sys Admin, I am NO script magician, and NO CI guru, I am a simple minded web dev, so pls pls be nice if its mindlessly easy to you, or if I stoopidly am missing the point, I don't mean to be all "Does You Haz the codes?" But I’ve basically spent 6 months dreaming that one day soon someone will post a nice clear simple blog entry with an "Idiots guide" that solves all my woes, and an hour later I am in deployment heaven again, but I am still waiting (or Googling badly), and its breaking my little Developer's heart :(

P.S. I promise that If I get a good answer here I'll do my bit for the fantastic SO community and spend at least 8 hours scouring for questions I might be able to help with and contributing back.

Answer 1

Great.

It seems the new SQL Server (Code Named Denali), along with the new SQL Server Developer Tools (Code Named Juneau), and specifically the 2.0 release of DAC projects may well have filled the gap between development and deployment to SQL Azure.

The new v2.0 of the DAC framework expands the set of supported objects to full support of SQL Azure schema objects and data types across all DAC services: extract, deploy, and upgrade

From SQL Azure Import/Export

Also see :

• Bob Beauchemins Blog Post suggesting schema upgrades on SQL Azure are now supported with the new DACImportExportCli.exe utility.
• Other suggestions the new DACs 2.0 solve the major issues with upgrade deployments to SQL Azure

And looks like it all should run side by side with current setup. Will check it out and update here on progress. Brilliant.