How to get a Fast .Net Http Request

Question

I need an Http request that I can use in .Net which takes under 100 ms. I'm able to achieve this in my browser so I really don't see why this is such a problem in code.

I've tried WinHTTP as well as WebRequest.Create and both of them are over 500ms which isn't acceptable for my use case.

Here are examples of the simple test I'm trying to pass. (WinHttpFetcher is a simple wrapper I wrote but it does the most trivial example of a Get Request that I'm not sure it's worth pasting.)

I'm getting acceptable results with LibCurlNet but if there are simultaneous usages of the class I get an access violation. Also since it's not managed code and has to be copied to bin directory, it's not ideal to deploy with my open source project.

Any ideas of another implementation to try?

    [Test]
    public void WinHttp_Should_Get_Html_Quickly()
    {
        var fetcher = new WinHttpFetcher();
        var startTime = DateTime.Now;          
        var result = fetcher.Fetch(new Uri("http://localhost"));
        var endTime = DateTime.Now;
        Assert.Less((endTime - startTime).TotalMilliseconds, 100);
    }
    [Test]
    public void WebRequest_Should_Get_Html_Quickly()
    {
        var startTime = DateTime.Now;
        var req = (HttpWebRequest) WebRequest.Create("http://localhost");
        var response = req.GetResponse();
        var endTime = DateTime.Now;
        Assert.Less((endTime - startTime).TotalMilliseconds, 100);
    }

Answer 1

When benchmarking, it is best to discard at least the first two timings as they are likely to skew the results:

• Timing 1: Dominated by JIT overhead i.e. the process of turning byte code into native code.
• Timing 2: A possible optimization pass for the JIT'd code.

Timings after this will reflect repeat performance much better.

The following is an example of a test harness that will automatically disregard JIT and optimization passes, and run a test a set number of iterations before taking an average to assert performance. As you can see the JIT pass takes a substantial amount of time.

JIT:410.79ms

Optimize:0.98ms.

Average over 10 iterations:0.38ms

Code:

[Test]
public void WebRequest_Should_Get_Html_Quickly()
{
    private const int TestIterations = 10;
    private const int MaxMilliseconds = 100;

    Action test = () =>
    {
       WebRequest.Create("http://localhost/iisstart.htm").GetResponse();
    };

    AssertTimedTest(TestIterations, MaxMilliseconds, test);
}

private static void AssertTimedTest(int iterations, int maxMs, Action test)
{
    double jit = Execute(test); //disregard jit pass
    Console.WriteLine("JIT:{0:F2}ms.", jit);

    double optimize = Execute(test); //disregard optimize pass
    Console.WriteLine("Optimize:{0:F2}ms.", optimize);

    double totalElapsed = 0;
    for (int i = 0; i < iterations; i++) totalElapsed += Execute(test);

    double averageMs = (totalElapsed / iterations);
    Console.WriteLine("Average:{0:F2}ms.", averageMs);
    Assert.Less(averageMs, maxMs, "Average elapsed test time.");
}

private static double Execute(Action action)
{
    Stopwatch stopwatch = Stopwatch.StartNew();
    action();
    return stopwatch.Elapsed.TotalMilliseconds;
}