Creating a rollback build for TeamCity

Question

We have a nightly TeamCity build which releases the latest code to our test website, restores the database to match production, then applies any schema and data changes we have in TFS. If this nightly build fails, the website is down until we manually fix the code and/or the database scripts and restart the build.

What I'd like is to automatically rollback to the last successful build so the website is available despite of any build break.

After spending a bit of time investigating, here's my proposed solution:

1. Nightly build runs, creating a new label in TFS (something like Nightly-build-{build number})
2. Create a new TeamCity build which triggers after the nightly build runs
3. Find the last successful nightly build number
4. Get the version relating to that build number's label in TFS
5. Rollback builds (it doesn't matter if the nightly build just finished successfully)

What I don't know is how to have the rollback build get the version based off a label.

Any help for this, or another solution, would be appreciated.

Cheers.

Answer 1

A judicious use of the TeamCity REST API might work here. I'm cribbing a bit from this question, which covers some of this same territory. You might could do something like this (and I'm just spitballing here):

1. Create your VerifyBuild configuration as per your Step 2.
2. Create a RollbackBuild configuration that can deploy from a given label, the build number of which is parameterized as %rollback.buildnumber%

In VerifyBuild:

1. Use the Rest API to list the most recent nightly production deployments
2. If the most recent deployment was a SUCCESS, then you're done.
3. If the most recent deployment was a FAILURE, then get the build number of the last successful build.
4. Use the Rest API to set the %rollback.buildnumber% of the RollbackBuild to the last successful build number.
5. Use the Rest API to queue a RollbackBuild.

I'm proposing this method because I don't know how you would dynamically fetch the correct label for RollbackBuild prior to the checkout, so I'm using VerifyBuild to pre-populate it.