NUnit and testing a Xamarin PCL

Question

I am developing a portable class library and iOS app where the data access is based on top of Paul Betts' ModernHttpClient, Fusillade, Refit and Akavache libraries. This works in the PCL and iOS apps but I am having an issue when creating unit tests against the PCL.

If I use the Resharper test running within Visual Studio 2015 all my tests pass but if I run the NUnit tests in Xamarin Studio about half the tests fail with TypeLoadExceptions. I have tried to determine which dll is causing me the problems but as far as I can see everything is as it should be.

PCL Class under test

   internal class ApiService<TRefitService> : IApiService<TRefitService> where TRefitService : IServiceContract
    {
        private readonly Lazy<TRefitService> background;
        private readonly Lazy<TRefitService> userInitiated;
        private readonly Lazy<TRefitService> speculative;

        private ApiService(){}

        public ApiService(string baseUrl)
        {
            ParameterGuard.ThrowIfNullOrEmpty(baseUrl, nameof(baseUrl));

            Func<string, RateLimitedHttpMessageHandler, TRefitService> createClient = (address, handler) =>
            {
                HttpClient client = new HttpClient(handler) {BaseAddress = new Uri(address)};
                return RestService.For<TRefitService>(client);
            };

            background = new Lazy<TRefitService>(() => createClient(baseUrl, new RateLimitedHttpMessageHandler(new NativeMessageHandler(), Priority.Background)));
            speculative = new Lazy<TRefitService>(() => createClient(baseUrl, new RateLimitedHttpMessageHandler(new NativeMessageHandler(), Priority.Speculative)));
            userInitiated = new Lazy<TRefitService>(() => createClient(baseUrl, new RateLimitedHttpMessageHandler(new NativeMessageHandler(), Priority.UserInitiated)));
        }

        /// <summary>
        /// Use for network requests that are running in the background.
        /// </summary>
        public TRefitService Background => background.Value; 

        /// <summary>
        /// Use to fetch data into a cache when a page loads. Expect that these requests will only get so far then give up and start failing.
        /// </summary>
        public TRefitService Speculative => speculative.Value;

        /// <summary>
        /// Use for network requests that are fetching data that the user is waiting on *right now*.
        /// </summary>
        public TRefitService UserInitiated => userInitiated.Value;
    }

Xamarin Studio Failing Test

        [Test()]
        public void PassingInANullBaseUrlThrowsAnException()
        {
            Assert.Throws<ArgumentNullException>(() => new ApiService<IAuthorisationApi>(null));
        }

Test Result

Expected: But was: (Could not load type 'ApiService`1' from assembly '/Users/206474978/Desktop/test/test/test/bin/Debug/xxx.xxx.xxx.dll'.) at NUnit.Framework.Assert.Throws (IResolveConstraint expression, NUnit.Framework.TestDelegate code, System.String message, System.Object[] args) <0x377c1c8 + 0x000e6> in :0

at NUnit.Framework.Assert.That (System.Object actual, IResolveConstraint expression, System.String message, System.Object[] args) <0x377c888 + 0x000bf> in :0 at NUnit.Framework.Assert.Throws (IResolveConstraint expression, NUnit.Framework.TestDelegate code, System.String message, System.Object[] args) <0x377c1c8 + 0x00137> in :0
at NUnit.Framework.Assert.Throws (System.Type expectedExceptionType, NUnit.Framework.TestDelegate code, System.String message, System.Object[] args) <0x377bfc8 + 0x0004f> in :0
at NUnit.Framework.Assert.Throws[T] (NUnit.Framework.TestDelegate code, System.String message, System.Object[] args) <0x377bf58 + 0x00033> in :0 at NUnit.Framework.Assert.Throws[T] (NUnit.Framework.TestDelegate code) <0x377bf08 + 0x0003b> in :0 at test.Test+ApiServiceTest.PassingInANullBaseUrlThrowsAnException () [0x00001] in /Users/206474978/Desktop/test/test/test/Test.cs:37 at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /private/tmp/source-mono-4.2.3/bockbuild-mono-4.2.0-branch/profiles/mono-mac-xamarin/build-root/mono-4.2.3/mcs/class/corlib/System.Reflection/MonoMethod.cs:295

Now this wouldn't normally be an issue as I would just use reshaper but I am building our Continuous Integration pipeline on top of Jenkins and running the tests via the nunit package within the Xamarin libraries which results in the build reporting failures.

Any idea what the problem is here?

UPDATE

Ok, after follow the tip from SushiHangover I used the latest version of NUnit test runner and got the following output. Seems like the errors are either TypeLoadException or BadImageFormatException but i'm unsure why. I checked the output for both projects and both are set to AnyCPU.

    [Test()]
public void PassingInAValidBaseAddressGivesAValidRefitUserInitiatedObject()
{
    var api = new ApiService<IAuthorisationApi>(BaseAddress);
    Assert.IsNotNull(api.UserInitiated);
}

    [Test()]
    public async Task PassingInAValidBaseAddressReturnsAValidAccountFromTheBackgroundObject()
    {
        //Arrange
        ApiService<IAuthorisationApi> service = = new ApiService<IAuthorisationApi>(BaseAddress);;
        //Act

        AuthorisationDto account = await service.Background.GetAuthorisationToken(logonPost, "ireland");
        //Assert
        Assert.IsTrue(account != null);
    }

Invalid type xxx.xxx.Core.Services.Lib.ApiService`1 for instance field xxx.xxx.Core.Tests.ApiServiceTest+c__async0:__0

7) Error : xxx.xxx.Core.Tests.ApiServiceTest.PassingInAValidBaseAddressGivesAValidRefitUserInitiatedObject
System.TypeLoadException : Failure has occurred while loading a type.
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) <0x3083b38 + 0x00093> in <filename unknown>:0 

8) Error : xxx.xxx.Core.Tests.ApiServiceTest.PassingInAValidBaseAddressReturnsAValidAccountFromTheBackgroundObject
System.BadImageFormatException : Could not resolve field token 0x040001ae
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) <0x3083b38 + 0x00093> in <filename unknown>:0 

UPDATE 2

Another tests fails with the below stacktrace, this looks like some issue with mono?

System.TypeLoadException : Could not load type 'ApiService`1' from assembly '/Users/206474978/Documents/Development/Projects/BRSMemberApp/UnitTests/BrsGolf.Members.Core.Tests/bin/Debug/Brs.Members.Core.dll'.

at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in /private/tmp/source-mono-4.2.3/bockbuild-mono-4.2.0-branch/profiles/mono-mac-xamarin/build-root/mono-4.2.3/mcs/class/corlib/System.Reflection/MonoMethod.cs:295

Answer 1

(This is related to older answer that I posted)

The Mono framework that is installed includes a relatively old version of NUnit (version 2.4.8). There are a number of issues/bugs in that version that range from incorrect exception trapping to solution/project parsing problems.

What we do for our OS-X builds (including Jenkins CI) is to install the latest NUnit via nuget to our solution root directory and use that installation for the local testing of the build artifacts.

Something like:

cd YourSolutionRootDir
nuget install nunit.runners
mono ./NUnit.ConsoleRunner.3.2.0/tools/nunit3-console.exe MyTestProject/bin/debug/except.dll

Answer 2

OK,

This was quite a nasty one. The problem was neither related directly to Mono or NUnit. Although the older version of NUnit was causing other issues.

Thanks to @SushiHangover for the tip that got me starting in resolving this problem.

Initially I broke down the unit tests to instantiate each dependencies of the APIService class that ultimately flagged an issue with the System.Net.Http dll over the iOS, PCL and Unit test projects. It appears that one of the libraries I was implementing was referencing v1.5.0.0 of this library causing a version mismatch over the projects, this was exacerbated by incorrect references (.Net, PCL etc). I resolved this by including an specific assembly binding in each projects app.config.

  <dependentAssembly>
    <assemblyIdentity name="System.Net.Http" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
    <bindingRedirect oldVersion="0.0.0.0-4.0.0.0" newVersion="4.0.0.0" />
  </dependentAssembly>

This solved around 80 failing unit tests but still left around 50 failing.

Further poking around flagged an issue with the versions of Akavache caching library across the PCL and unit test projects which resolved another 40 unit tests.

Finally there was an issue with running Telerik's JustMock library in the Unit Test project where parts of the library does not appear to be supported when testing a PCL. I resolved this be swapping out JustMock for Moq and refactoring some tests. This resolved the remaining failing unit testing with Xamarin Studio.

I then ran these tests on the build server and get 80 passing tests and 87 not run. Damn.

To resolve this issue I swapped out the default test runner in Xamarin v2.4.8 for the most recent one downloaded and now all unit test pass on the build server.

To cut a long story short check you references when you get these kind of issues.