Detecting breaking changes in .NET code using TFS?

Question

I would like to detect breaking changes in .NET code (specifically C#) whenever TFS builds a solution. If there are any breaking changes (such as outlined in "A definite guide to API-breaking changes in .NET") between the code being checked in and the version in the most recent successful build, I would like to know about it. A breaking change needn't cause the build to fail. Short of writing an app that uses reflection to compare two versions of the same assembly, how can this be done?

Answer 1

To elaborate a bit on James and Adam answers, I'd like to provide details concerning detecting breaking changes with NDepend and its code query and rule capabilities. Disclaimer: I am one of the developers of the tool

NDepend has evolved and its query language as well. If you download NDepend trial and analysis the two versions of your code base where you'd like to search for breaking change, have a look in the default code rules group API Breaking Changes for the following CQLinq rules.

• API Breaking Changes: Types
• API Breaking Changes: Methods
• API Breaking Changes: Fields
• API Breaking Changes: Interfaces and Abstract Classes
• Broken serializable types
• API: New publicly visible types
• API: New publicly visible methods
• API: New publicly visible fields

Executing one of these code rule looks like for example (diff between NUnit v2.5.8 and v2.5.3):

Answer 2

Yes, I would (and do) use NDepend for this. I work on a product which provides an extendable API for developers. As such we need to make sure that between releases, we do not remove functionality that those developers may depend on. The flip-side, is that we need the flexibility to grow the product without massive constraints around reversioning.

Some things you will want to consider.

1. Changing the version of a referenced DLL should be considered a breaking change.
2. removing/changing members breaks backwards compatibility.
3. adding members breaks forwards compatibility (some people just consider 'added members' as safe, but it does have a risk associated).
4. Change file version with every build, you will need it at some point.
5. Consider writing contracts that define your 'public API'. These will be the members that you need to support outside of the organisation. Think of them as interoperability boundaries. It then allows your implementation classes to have public members, which arent in the API (hence considered 'unsupported'), so you can change them without worrying about breaking the extensibility API. Extending the API consists of writing a new interface (with a version number in the interface name) which DOES NOT derive from the prior version of interface (derivation prevents you from fully deprecating members, and creates hell when it comes time to implement multiple interface versions in a single class.
6. Dont forget about Attributes, changes to them may not break static compatiblity, but could affect the runtime.

Answer 3

Unit tests. They provide a way to assert 'this is what client code expects'. You can have TFS run unit tests when you build.

Answer 4

Patrick Smacchia of NDepend fame posted about this ~3.5 years ago.

http://codebetter.com/patricksmacchia/2008/01/20/avoid-api-breaking-changes/

He mentions LibCheck and (obviously) NDepend, and a comment mentions one more.

Since it's been 3.5+ years since, there may be better options available these days (LibCheck is well over 6 years old), but those should be a start.