How to programmatically run unit tests with Gallio and MBUnit?

Question

I'm trying to programmatically check my unit tests are passing as part of my deployment process. The application uses MBunit and Gallio for it's unit testing framework.

Here's my code:

var setup = new Gallio.Runtime.RuntimeSetup();
setup.AddPluginDirectory(@"C:\Program Files\Gallio\bin");

using (TextWriter tw = new StreamWriter(logFilename))
{
    var logger = new Gallio.Runtime.Logging.TextLogger(tw);
    RuntimeBootstrap.Initialize(setup, logger);

    TestLauncher launcher = new TestLauncher();                
    launcher.AddFilePattern(dllToRunFilename);
    TestLauncherResult result = launcher.Run();
}

Here's the test which is contained in the DLL I'm loading (I've validated this works with the Icarus test runner):

public class Tests
    {
        [Test]
        public void Pass()
        {            
            Assert.IsTrue(true);
        }

        [Test]
        public void Fail()
        {
            Assert.Fail();
        }
    }

When I run the application I get the following values in results

Which is incorrect as there are indeed tests to run! The log file has the following in it

Disabled plugin 'Gallio.VisualStudio.Shell90': The plugin enable condition was not satisfied. Please note that this is the intended behavior for plugins that must be hosted inside third party applications in order to work. Enable condition: '${process:DEVENV.EXE_V9.0} or ${process:VSTESTHOST.EXE_V9.0} or ${process:MSTEST.EXE_V9.0} or ${framework:NET35}'. Disabled plugin 'Gallio.VisualStudio.Tip90': The plugin depends on another disabled plugin: 'Gallio.VisualStudio.Shell90'.

How do I resolve this issue and find the results to the tests?

Answer 1

This works for me, note I used this GallioBundle nuget to get gallio and mbunit, so perhaps there is some difference to what you have installed.

The log messages regarding plugins are expected, those plugins wont work if you are self-hosting the Gallio runtime.

using System;
using System.IO;
using Gallio.Runner;
using Gallio.Runtime;
using Gallio.Runtime.Logging;
using MbUnit.Framework;

public static class Program
{
    public static void Main()
    {
        using (TextWriter tw = new StreamWriter("RunTests.log"))
        {
            var logger = new TextLogger(tw);
            RuntimeBootstrap.Initialize(new RuntimeSetup(), logger);

            TestLauncher launcher = new TestLauncher();
            launcher.AddFilePattern("RunTests.exe");
            TestLauncherResult result = launcher.Run();
            Console.WriteLine(result.ResultSummary);
        }
    }
}

public class Tests
{
    [Test]
    public void Pass()
    {
        Assert.IsTrue(true);
    }

    [Test]
    public void Fail()
    {
        Assert.Fail();
    }
}

Tested like this:

› notepad RunTests.cs
› nuget.exe install -excludeversion GallioBundle
  Installing 'GallioBundle 3.4.14.0'.
  Successfully installed 'GallioBundle 3.4.14.0'.
› cd .\GallioBundle\bin
› csc ..\..\RunTests.cs /r:Gallio.dll /r:MbUnit.dll
  Microsoft (R) Visual C# Compiler version 12.0.21005.1
  for C# 5
  Copyright (C) Microsoft Corporation. All rights reserved.    
› .\RunTests.exe
  2 run, 1 passed, 1 failed, 0 inconclusive, 0 skipped