Unit test ReSharper and NUnit give different results

Question

Using ReSharper 8.2 on local computer and NUnit 2.6.3 on build server found there were some test which passed in ReSharper and failed in NUnit. Installed NUnit locally and same results so it is not a difference between computers. Two of my colleagues ran the same tests and had same results so doesn't seem something messing my computer.

A simplified version of the tests:

[Test]
public void Test_UrlQueryString()
{
    var urlInput = "http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE";
    var uri = new Uri(urlInput);

    Assert.AreEqual(urlInput, uri.ToString());
}

[Test]
public void Test_Dot()
{
    var urlInput = "http://www.domain.com/page-with-dot.?p=google";
    var uri = new Uri(urlInput);

    Assert.AreEqual(urlInput, uri.ToString());
}

ReSharper output is all green. The output from NUnit:

Runtime Environment -
   OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
  CLR Version: 4.0.30319.18444 ( Net 4.5 )

ProcessModel: Default    DomainUsage: Single
Execution Runtime: net-4.5
...................F.F.........
Tests run: 29, Errors: 0, Failures: 2, Inconclusive: 0, Time: 0.576769973208475 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

Errors and Failures:

  1) Test Failure : Test.OrganicTest.Test_Dot
     Expected string length 45 but was 44. Strings differ at index 35.
  Expected: "http://www.domain.com/page-with-dot.?p=google"
  But was:  "http://www.domain.com/page-with-dot?p=google"
  ----------------------------------------------^

  2) Test Failure : Test.OrganicTest.Test_UrlQueryString
     Expected string length 87 but was 83. Strings differ at index 76.
  Expected: "...-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE"
  But was:  "...-with-querystring?url=https://www.domain2.com/page?p=PEPE"
  ----------------------------------------------------------------^

ReSharper seems to be using the same version of NUnit (Built-in NUnit 2.6.3)

Does anyone knows what how to fix that? Is it a bug in ReSharper or NUnit?

Answer 1

Originally, it is a bug in .NET Framework. The URI scaping behaviour depends on environment (.NET Framework version and config). The bug was fixed in .NET Framework 4.5. For example, let's take the code

var uri1 = "http://www.domain.com/page-with-dot.?p=google";
var uri2 = "http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE";
Console.WriteLine(new Uri(uri1).ToString());
Console.WriteLine(new Uri(uri2).ToString()); 

Output in .NET Framework 3.5:

http://www.domain.com/page-with-dot?p=google
http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page?p=PEPE

Output in .NET Framework 4.5:

http://www.domain.com/page-with-dot.?p=google
http://www.domain.com/page-with-querystring?url=https://www.domain2.com/page%3Fp%3DPEPE

Also, you can make some changes in escaping behaviour with your .config file. For example, there is an option for .NET 4.0 and slash escaping:

<configuration>
  <uri>
    <schemeSettings>
      <add name="http" 
           genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
    </schemeSettings>
  </uri>
</configuration>

I think, in your case, ReSharper and NUnit just use different configs. Although, it is really strange, that NUnit received old uri escaping results for .NET 4.5. Maybe NUnit use some microsoft libraries from .NET 4.0. Also, you can fail the test in ReSharper by manual set the Framework option to CLR4.0. Anyway, you should rewrite tests with environment independent logic.

Useful links:

• MS Connect 511010: Erroneous URI parsing for encoded, reserved characters, according to RFC 3986
• Mono Bug 16960
• StackOverflow: Getting a Uri with escaped slashes on mono
• StackOverflow: GETting a URL with an url-encoded slash
• Mono 3.10.0 release notes
• Mike Hadlow: How to stop System.Uri un-escaping forward slash characters
• Arnout's Eclectica: URL-encoded slashes in System.Uri
• About slash escaping in .NET (In Russian)