Automatically Unit Test Example Code

Question

My team is responsible for the development of an API for a large system that we also write. We need to provide example code so that other developers using our API can learn how to use it. We have been documenting the code using the xml document comments. eg.

/// <summary>Summary here</summary>
/// <example>Here is an example  <code>example code here</code> </example>
public void SomeFunction() 

We then use Sandcastle and build the help files we need (chm and an online website).

It is quite embarrassing when the example code doesnt work, and this is usually because some functionality has changed or a simple error.

Has anyone ever done something like this, but also configured unit tests to run on the example code so that they are known to work during the build?

Answer 1

I would suggest using a special bit of markup in your XML which says, "Grab the code sample from this place". It would refer to a normal C# file which can be run with unit tests. To take your example, you might have:

/// <summary>Summary here</summary>
/// <example>Here is an example
/// <code>!!sourcefile:SomeClassTest.cs#SomeFunction!!</code></example>
public void SomeFunction()

Your unit tests run as normal, and then insert a build step between "create XML" and "run Sandcastle" you'd replace each "file token" with the appropriate contents. There may even be hooks you could put into Sandcastle to do this at doc generation time - I don't know enough about Sandcastle to know for sure.

It's ugly to invent your own markup of course, but it should work.

Of course, this assumes that the code examples are easily unit testable - some may not be (if they're dealing with resources etc). At least you'd know it compiles though :)

Answer 2

Yes, sandcastle supports this and it's great to maintain the correctness of examples. You can point to a code region like this:

   /// <summary>
   /// Gizmo which can act as client or server.
   /// </summary>
   /// <example>
   /// The following example shows how to use the gizmo as a client:
   /// <code lang="cs"
   ///    source="..\gizmo.unittests\TestGizmo.cs"
   ///    region="GizmoClientSample"/>
   /// </example>
   public class Gizmo

You can then use some test code in TestGizmo.cs as an example by enclosing it in a region:

[Test]
public GizmoCanActAsClient()
{
   #region GizmoClientSample
   Gizmo gizmo = new Gizmo();
   gizmo.ActAsClient();
   #endregion
}

Caveat: If you move or rename the test file, you will only get an error about this when you try to regenerate the documentation with sandcastle.