TFS 2008/2010 vs Jenkins for Continuous Integration

Question

Does anyone have specific experiences with using TFS 2008/2010 AND Jenkins for Continuous Integration (CI)? We are trying to decide which CI server to use. Our team works exclusively in Microsoft .NET/Visual Studio 2010/C#. We have the following requirements:

1. Automatically build our web project on every checkin.
2. Run unit tests with each build.
3. Automatically deploy green builds to development and/or test environments.
4. Provide pretty reports.
5. Provide build/deployment notifications via email.

I realize that installing a tool won't necessarily give us this functionality out-of-the-box and that we will have to integrate with other tools like MSBuild to achieve this.

I'm looking for specific features that Jenkins has that TFS 2008/2010 does not or vice versa. Also which is easier to maintain, use, etc.

Answer 1

I would highly recommend using Jenkins - it will do all of your requirements out of the box apart from possibly #3, but if you can script your deployments then it can do that as well.

Here are some links to help you get your builds up and running:

Blog about doing .NET builds in Jenkins

Jenkins Windows installers

Installing the Jenkins master and slaves as Windows services

Disclaimer: I have no experience with TFS, but I think open solutions are nearly always more flexible and extensible (and cheaper !) than proprietary products.

Answer 2

Late to this game, but I have used both TFS 2010 and Jenkins for CI. TFS 2010 has minimum set of CI tools in it. However, when you want to create a CI pipeline, it's a completely different story while Jenkins can easily create the pipeline.

If you are looking at only CI for one build either one should work. However, when it comes to the entire pipeline, Jenkins is way to go. With TFS it can be done, but Jenkins is better choice.

Here's quick bullet points:

TFS:

• With a build definition you can compile, execute tests, return changeset/workitems, send an email when a build is broken

• natural integration with visual studio

• extremely hard to create CI pipeline. Requires custom handler and extensive workflow work. Not as intuitive as creating a build definition.

• Because of the 3rd bullet, it's not easy to maintain/customize/scale CI pipeline

Jenkins:

• Need to create a msbuild config file for CI, which is not much pain comparing to creating CI pipeline using TFS. However, TFS gives better/easier tool to create a build definition. however, it is not bad creating config file for msbuild for a project.

• Creating a CI pipeline is very easy. Just chain them using upstream/downstream jenkins job trigger and passing an artifact from previous job.

• Since Jenkins is very flexible, it is easy to create a jenkins plugin to meet your own needs and provide it to opensource community :)

In summary, if you need complete automated build, test, and deployment system go with Jenkins. If you just need only build and test, TFS might give you an edge over Jenkins.