Silverlight unit testing integration in VS 2010?

Question

I'm currently using the Silverlight Unit Test Framework, but I'd prefer to run tests directly in VS2010. I'm curious to know what approaches and tools everyone else uses.

I'm using Silverlight 4 with Prism and the MVVM pattern, and I'm specifically interested in integrated Silverlight unit test support in VS 2010 that I can use with my ViewModel unit tests. I'm using dependency injection with Unity, and I am writing unit tests by mocking the calls in my WCF layer using Moq for Silverlight. I am not even looking at integration tests at the moment, but even in a simple unit test which tests a single ViewModel command, the service request to my mocked service layer can take around 50 milliseconds. Therefore support for asynchronous tests is important to me.

The issue I'm raising here is not related to View testing, which I have handled with some success in the past using System.Windows.Automation.Peers, and - although I have not used it yet - could possibly now handle more easily with the support in VS 2010 Feature Pack 2 (which appears to be targeted at automation / playback of UI tests from what I gather).

I should mention that my findings from the products I've looked at and used so far are as follows:

• Silverlight Unit Test Framework - I currently use this, and it's great as far as it goes, but its limitations are (a) it is not integrated with Visual Studio; and (b) if you don't want to run all tests, you are limited to the crude tag expression filter.
• StatLight - very nice. I currently use this, and have used it since v0.9 when targeting Silverlight 3 on a previous project. Being a command-line tool, it can be integrated with a continuous integration server - which certainly handles another required scenario. But it is of no use directly in terms of Visual Studio integration during the development process.
• Unit Test Result Viewer for Silverlight (Visual Studio extension on the Visual Studio gallery) - looks promising, but its limitations are (a) currently fails to find projects which are located in solution folders, rather than under the solution root; and (b) runs all tests in a given assembly (via StatLight), with no apparent ability to run a specific test, or a selection of tests.
• Einar Ingebrigtsen's Silverlight Unit Test Runner for ReSharper, which later became Odin - ahead of the game (it first appeared in 2008), but the limitation is that it appears that this project is no longer maintained (most recent update is Apr 2009).
• AgUnit ReSharper plugin ( http://agunit.codeplex.com/ ) - looks excellent initially. After downloading the source code for it and building the latest (bug-fixed) version to work against ReSharper 5.1, I was very encouraged. But unfortunately it does not handle asynchronous tests. This is a design limitation with the threading, so it does not matter whether you try to use the asynchronous support that is built into Silverlight Unit Test Framework (Microsoft.Silverliht.Testing.SilverlightTest base class), or whether you're using AutoResetEvent or anything else. This has been noted by the coordinator on the project's discussion forum on CodePlex. This is a massive limitation.
• TestDriven.NET 3.0 - appears to have support for Silverlight 4.0 tests at first glance, but the limitation (I suspect) is Silverlight 4 "assembly portability" (i.e. the 5 dependent assemblies that are portable between SL4 and .NET 4). Certainly, when I tried using it with a simple POC, it crashed my instance of VS 2010.

Perhaps I've missed something here - I wonder if anyone in the community has any better ideas for Silverlight unit testing?

Answer 1

I use the Silverlight Unit test framework, AgUnit, and RX with a mock IScheduler provider to make my unit tests single threaded :)

Answer 2

UPDATE:

I'd already settled on using StatLight for my continuous integration server, but I was looking for a solution to allow me to run asynchronous Silverlight unit tests directly in VS2010 during development.

Inspired by Rob Fonseca-Ensor's suggestion to use Rx (see his separate answer on this page), I took another look at the issue. That led me to find a non-Rx solution to the problem of using AgUnit to run async Silverlight unit tests.

The solution I think I will use - at least for now - is the combination of:

• Silverlight Unit Test Framework
• ReSharper 5.1
• AgUnit ReSharper plugin
• [Mock async pattern]

My ViewModels have bespoke service classes injected into them which provide abstractions of my (auto-generated) WCF service reference classes. To help provide access to the WCF service methods, my bespoke service classes also rely on another common class which wraps the async pattern. For my unit tests, I was already mocking my WCF service reference classes and my bespoke service classes with Moq - but I hadn't looked at mocking my async pattern wrapper class.

So I decided to mock my async pattern wrapper class as well. The attraction of doing this was that I thought I might then be able to use Rx with a mock IScheduler (as Rob suggested in his answer) in all my mocks, while leaving my real classes free of any references to Rx (which is a requirement because for this project at the place I'm working, I need to keep Rx out of any code which gets deployed to a production environment). However, once I'd mocked the wrapper class, I realised that I didn't even need Rx, and that there was an even simpler solution - which I should have really looked at before. It's pretty trivial - all I really needed to do was mock the wrapper class and ensure that it called Invoke on the callback operations, rather than calling BeginInvoke. This ultimately prevents the callbacks which work fine when running under the Silverlight Unit Test Framework in a browser session from going into a black hole when the unit tests are run by AgUnit within VS2010. (Allowing the callbacks to go into black hole when using AgUnit would of course have prevented the individual tests from completing properly, and could have led to either timeouts or - worse still - false positives in the test results.)

In the absence of better advice from any subsequent answers to this question, this appears to be the easiest way for me to handle this scenario - allowing my async unit tests to run in AgUnit without requiring Rx in my production code.

For anyone else not facing company-imposed restrictions on deploying code that uses Rx (which is still a DevLabs project) to production, I think Rob's solution would definitely be worth looking at.