Recommendations for Continuous integration for Mercurial/Kiln + MSBuild + MSTest

Question

We have our source code stored in Kiln/Mercurial repositories; we use MSBuild to build our product and we have Unit Tests that utilize MSTest (Visual Studio Unit Tests).

What solutions exist to implement a continuous integration machine (i.e. Build machine).

The requirements for this are:

• A build should be kicked of when necessary (i.e. code has changed in the Repositories we care about)
• Before the actual build, the latest version of the source code must be acquired from the repository we are building from
• The build must build the entire product
• The build must build all Unit Tests
• The build must execute all unit tests
• A summary of success/failure must be sent out after the build has finished; this must include information about the build itself but also about which Unit Tests failed and which ones succeeded.
• The summary must contain which changesets were in this build that were not yet in the previous successful (!) build
• The system must be configurable so that it can build from multiple branches(/Repositories).

Ideally, this system would run on a single box (our product isn't that big) without any server components.

What solutions are currently available? What are their pros/cons? From the list above, what can be done and what cannot be done?

Thanks

Answer 1

TeamCity, from JetBrains, the makers of ReSharp, will do all of that. You will have to configure it for what specifically it means to "build your product", but you can configure up everything you specified with it.

The software can alert you to failed builds, even down to alerting only the person responsible for checking in code that broke the build. It even comes with handy web pages you can view to see only your own changes, which builds they've been through successfully, which ones are pending, and which ones are currently being executed.

Since it is a distributed product, you can make it grow with your organization and product. If at some point you discover that you're waiting for the build to complete too much, because a lot of builds are being queued up, you can add more build agents. The build agents are basically separate client programs you install on additional machines, that execute the actual build configurations.

It comes in two flavors, the professional version and the enterprise version. The professional version is free, can contain up to 20 build configurations, 20 users, and 3 build agents. The enterprise version has unlimited users and build configurations, and you can also use LDAP based security (think domain verified users.) There's also some other bonuses from the enterprise version. You can also buy licenses for more build agents if you need more than the initial 3.

Now, if "no server components" means you don't want it to act like a web server, you're going to be hard pressed to find something that will react to your commits.

However, if you mean that you don't want to have to install a server OS, then TeamCity can work on workstation versions of Windows as well. That isn't to say that you shouldn't consider setting up a proper server for it, but it will run on a workstation if that is what you require.

Answer 2

Our product BuildMaster does all of the things you listed by design and there is a free, somewhat limited edition (e.g. you can only have a limited number of issue tracking providers integrate with it, the database change script packaging tool isn't included in the free version, etc.) for 5 users or fewer.

Answer 3

What you've described is the basics of a CI Tool, so every CI Tool should be OK.
I use Cruise Control.NET but it is bugged with Mercurial and is not very straightforward at first glance. I am nevertheless happy with it. Other tools that come in my mind are Hudson, Team Build (from TFS) and TeamCity.

I have not tried other tools but you can see pros/cons here :

• TeamCity vs CC.net
• Hudson vs CC.net, Link 1 and Link 2
• CC.net vs TFS

EDIT : I forgot to mention that Hudson and Cruise Control.net are Open Source project, you can easily write plugins and patches to your install.

EDIT² : Mercurial bugs seem to be fixed in the upcoming 1.6 version of ccnet (changes commited to the trunk this week).