How can I test the backward compatibility of API between .net assemblies

Question

I have an assembly that provides an API and is used by some other assemblies. I need to verify that a newer version of API dll is still compatible with the older assemblies that were using the older version of API.

I've found a couple of questions that ask the same, but there are no answers that resolve my problem:

• Tool to verify compatibility of a public APIs
• Tool for backwards compatibility for the C#/.NET API

Suggested tools can only compare two assemblies and say if there are possible breaking changes in API, but not if the newest API really breaks the older assembly that uses it. I'd like to find a tool or write a test that will be able to check whether each of the older dlls can work with my new API dll.

As for the changes in API more likely that I will only extend it, but even though it still can break the code in older assemblies. Some of the examples of such changes can be found here:

• A definite guide to API-breaking changes in .NET
• .NET: with respect to AssemblyVersion, what defines binary compatibility?

For now the only solution I see is to compile the source code of the older assemblies with the newest API, but I would like to do it only with assemblies and add them as part of my unit tests. Is there any better way I can handle that?

edit:

I'm looking for a tool that will be able to automate the process of verifying the backward compatibility between .net assemblies. (command line or with some api too)

Answer 1

What you want is to do a diff and generate a the list of breaking changes. Then you want to search if of your assemblies does use any of the broken APIs. You can do this with ApiChange tool to do the diff and to find any affected users of it.

To make it more concrete. If you have removed a method from an interface then you need to find all implementers and users of this method in classes which uses the interface method or any class that does implement this method.

ApiChange can search for implementers and users of specific methods on the command line with the commands -whoimplementsinterface and -whousesmethod. It is not automated at the command line but you can directly use the ApiChange.Api.dll to automate this queries.

Edit1:

I just forgot: The ApiChange tool has actually the functionality you are interested in already. It is the option

-ShowrebuildTargets -new -old [-old2 ] -searchin

We did use it in our department with good results. The only gotcha are the XML Intellisense files. If another target does not use the removed method but references it inside the XmlDoc the compiler will write a warning that a non existing method was referenced. This is quite hard to catch and would involve to parse the intellisense docu files as well. But this is quite an edge case.

Answer 2

I've spent the day looking around for an answer to this. It seems like the tools referenced on the related (unhelpfully closed) questions are now dead or as good as. But I've just taken a look at Telerik's assembly diff tool JustAssembly and this looks much better than rolling your own, which, if you look at their library seems to be a whole heap of work and likely to go wrong.

They have a UI which isn't of that much help from the point of view of integrating into your CI build, it is pretty basic, but you can build the library from source, which I've just done and the library looks like it has everything you need to get yourself up and running pretty quickly.