Avoiding the overhead of C# virtual calls

Question

I have a few heavily optimized math functions that take 1-2 nanoseconds to complete. These functions are called hundreds of millions of times per second, so call overhead is a concern, despite the already-excellent performance.

In order to keep the program maintainable, the classes that provide these methods inherit an IMathFunction interface, so that other objects can directly store a specific math function and use it when needed.

public interface IMathFunction {   double Calculate(double input);   double Derivate(double input); }  public SomeObject {   // Note: There are cases where this is mutable   private readonly IMathFunction mathFunction_;     public double SomeWork(double input, double step)   {     var f = mathFunction_.Calculate(input);     var dv = mathFunction_.Derivate(input);     return f - (dv * step);   } } 

This interface is causing an enormous overhead compared to a direct call due to how the consuming code uses it. A direct call takes 1-2ns, whereas the virtual interface call takes 8-9ns. Evidently, the presence of the interface and its subsequent translation of the virtual call is the bottleneck for this scenario.

I would like to retain both maintainability and performance if possible. Is there a way I can resolve the virtual function to a direct call when the object is instantiated so that all subsequent calls are able to avoid the overhead? I assume this would involve creating delegates with IL, but I wouldn't know where to start with that.

Answer 1

So this has obvious limitations and should not be used all the time anywhere you have an interface, but if you have a place where perf really needs to be maximized you can use generics:

public SomeObject<TMathFunction> where TMathFunction: struct, IMathFunction  {   private readonly TMathFunction mathFunction_;    public double SomeWork(double input, double step)   {     var f = mathFunction_.Calculate(input);     var dv = mathFunction_.Derivate(input);     return f - (dv * step);   } } 

And instead of passing an interface, pass your implementation as TMathFunction. This will avoid vtable lookups due to an interface and also allow inlining.

Note the use of struct is important here, as generics will otherwise access the class via the interface.

Some implementation:

I made a simple implementation of IMathFunction for testing:

class SomeImplementationByRef : IMathFunction {     public double Calculate(double input)     {         return input + input;     }      public double Derivate(double input)     {         return input * input;     } } 

... as well as a struct version and an abstract version.

So, here's what happens with the interface version. You can see it is relatively inefficient because it performs two levels of indirection:

    return obj.SomeWork(input, step); sub         esp,40h   vzeroupper   vmovaps     xmmword ptr [rsp+30h],xmm6   vmovaps     xmmword ptr [rsp+20h],xmm7   mov         rsi,rcx vmovsd      qword ptr [rsp+60h],xmm2   vmovaps     xmm6,xmm1 mov         rcx,qword ptr [rsi+8]          ; load mathFunction_ into rcx. vmovaps     xmm1,xmm6   mov         r11,7FFED7980020h              ; load vtable address of the IMathFunction.Calculate function. cmp         dword ptr [rcx],ecx   call        qword ptr [r11]                ; call IMathFunction.Calculate function which will call the actual Calculate via vtable. vmovaps     xmm7,xmm0 mov         rcx,qword ptr [rsi+8]          ; load mathFunction_ into rcx. vmovaps     xmm1,xmm6   mov         r11,7FFED7980028h              ; load vtable address of the IMathFunction.Derivate function. cmp         dword ptr [rcx],ecx   call        qword ptr [r11]                ; call IMathFunction.Derivate function which will call the actual Derivate via vtable. vmulsd      xmm0,xmm0,mmword ptr [rsp+60h] ; dv * step vsubsd      xmm7,xmm7,xmm0                 ; f - (dv * step) vmovaps     xmm0,xmm7   vmovaps     xmm6,xmmword ptr [rsp+30h]   vmovaps     xmm7,xmmword ptr [rsp+20h]   add         rsp,40h   pop         rsi   ret   

Here's an abstract class. It's a little more efficient but only negligibly:

        return obj.SomeWork(input, step);  sub         esp,40h    vzeroupper    vmovaps     xmmword ptr [rsp+30h],xmm6    vmovaps     xmmword ptr [rsp+20h],xmm7    mov         rsi,rcx    vmovsd      qword ptr [rsp+60h],xmm2    vmovaps     xmm6,xmm1    mov         rcx,qword ptr [rsi+8]           ; load mathFunction_ into rcx.  vmovaps     xmm1,xmm6    mov         rax,qword ptr [rcx]             ; load object type data from mathFunction_.  mov         rax,qword ptr [rax+40h]         ; load address of vtable into rax.  call        qword ptr [rax+20h]             ; call Calculate via offset 0x20 of vtable.  vmovaps     xmm7,xmm0    mov         rcx,qword ptr [rsi+8]           ; load mathFunction_ into rcx.  vmovaps     xmm1,xmm6    mov         rax,qword ptr [rcx]             ; load object type data from mathFunction_.  mov         rax,qword ptr [rax+40h]         ; load address of vtable into rax.  call        qword ptr [rax+28h]             ; call Derivate via offset 0x28 of vtable.  vmulsd      xmm0,xmm0,mmword ptr [rsp+60h]  ; dv * step  vsubsd      xmm7,xmm7,xmm0                  ; f - (dv * step)  vmovaps     xmm0,xmm7  vmovaps     xmm6,xmmword ptr [rsp+30h]    vmovaps     xmm7,xmmword ptr [rsp+20h]    add         rsp,40h    pop         rsi    ret   

So both an interface and an abstract class rely heavily on branch target prediction to have acceptable performance. Even then, you can see there's quite a lot more going into it, so the best-case is still relatively slow while the worst-case is a stalled pipeline due to a mispredict.

And finally here's the generic version with a struct. You can see it's massively more efficient because everything has been fully inlined so there's no branch prediction involved. It also has the nice side effect of removing most of the stack/parameter management that was in there too, so the code becomes very compact:

    return obj.SomeWork(input, step); push        rax   vzeroupper   movsx       rax,byte ptr [rcx+8]   vmovaps     xmm0,xmm1   vaddsd      xmm0,xmm0,xmm1  ; Calculate - got inlined vmulsd      xmm1,xmm1,xmm1  ; Derivate - got inlined vmulsd      xmm1,xmm1,xmm2  ; dv * step vsubsd      xmm0,xmm0,xmm1  ; f -  add         rsp,8   ret   

Answer 2

I would assign the methods to delegates. This allows you to still program against the interface, while avoiding the interface method resolution.

public SomeObject {     private readonly Func<double, double> _calculate;     private readonly Func<double, double> _derivate;      public SomeObject(IMathFunction mathFunction)     {         _calculate = mathFunction.Calculate;         _derivate = mathFunction.Derivate;     }      public double SomeWork(double input, double step)     {         var f = _calculate(input);         var dv = _derivate(input);         return f - (dv * step);     } } 

---

In response to @CoryNelson's comment I made tests so see what the impact really is. I have sealed the function class, but this seems to make absolutely no difference since my methods are not virtual.

Test Results (mean time of 100 million iterations in ns) with the empty method time subtracted in braces:

Empty Work method: 1.48
Interface: 5.69 (4.21)
Delegates: 5.78 (4.30)
Sealed Class: 2.10 (0.62)
Class: 2.12 (0.64)

The delegate version time is about the same as for the interface version (the exact times vary from test execution to test execution). While working against the class is about 6.8 x faster (comparing times minus the empty work method time)! This means that my suggestion to work with delegates was not helpful!

What surprised me was, that I expected a much longer execution time for the interface version. Since this kind of test does not represent the exact context of the OP's code, its validity is limited.

static class TimingInterfaceVsDelegateCalls {     const int N = 100_000_000;     const double msToNs = 1e6 / N;      static SquareFunctionSealed _mathFunctionClassSealed;     static SquareFunction _mathFunctionClass;     static IMathFunction _mathFunctionInterface;     static Func<double, double> _calculate;     static Func<double, double> _derivate;      static TimingInterfaceVsDelegateCalls()     {         _mathFunctionClass = new SquareFunction();         _mathFunctionClassSealed = new SquareFunctionSealed();         _mathFunctionInterface = _mathFunctionClassSealed;         _calculate = _mathFunctionInterface.Calculate;         _derivate = _mathFunctionInterface.Derivate;     }      interface IMathFunction     {         double Calculate(double input);         double Derivate(double input);     }      sealed class SquareFunctionSealed : IMathFunction     {         public double Calculate(double input)         {             return input * input;         }          public double Derivate(double input)         {             return 2 * input;         }     }      class SquareFunction : IMathFunction     {         public double Calculate(double input)         {             return input * input;         }          public double Derivate(double input)         {             return 2 * input;         }     }      public static void Test()     {         var stopWatch = new Stopwatch();          stopWatch.Start();         for (int i = 0; i < N; i++) {             double result = SomeWorkEmpty(i);         }         stopWatch.Stop();         double emptyTime = stopWatch.ElapsedMilliseconds * msToNs;         Console.WriteLine($"Empty Work method: {emptyTime:n2}");          stopWatch.Restart();         for (int i = 0; i < N; i++) {             double result = SomeWorkInterface(i);         }         stopWatch.Stop();         PrintResult("Interface", stopWatch.ElapsedMilliseconds, emptyTime);          stopWatch.Restart();         for (int i = 0; i < N; i++) {             double result = SomeWorkDelegate(i);         }         stopWatch.Stop();         PrintResult("Delegates", stopWatch.ElapsedMilliseconds, emptyTime);          stopWatch.Restart();         for (int i = 0; i < N; i++) {             double result = SomeWorkClassSealed(i);         }         stopWatch.Stop();         PrintResult("Sealed Class", stopWatch.ElapsedMilliseconds, emptyTime);          stopWatch.Restart();         for (int i = 0; i < N; i++) {             double result = SomeWorkClass(i);         }         stopWatch.Stop();         PrintResult("Class", stopWatch.ElapsedMilliseconds, emptyTime);     }      private static void PrintResult(string text, long elapsed, double emptyTime)     {         Console.WriteLine($"{text}: {elapsed * msToNs:n2} ({elapsed * msToNs - emptyTime:n2})");     }      [MethodImpl(MethodImplOptions.NoInlining)]     private static double SomeWorkEmpty(int i)     {         return 0.0;     }      [MethodImpl(MethodImplOptions.NoInlining)]     private static double SomeWorkInterface(int i)     {         double f = _mathFunctionInterface.Calculate(i);         double dv = _mathFunctionInterface.Derivate(i);         return f - (dv * 12.34534);     }      [MethodImpl(MethodImplOptions.NoInlining)]     private static double SomeWorkDelegate(int i)     {         double f = _calculate(i);         double dv = _derivate(i);         return f - (dv * 12.34534);     }      [MethodImpl(MethodImplOptions.NoInlining)]     private static double SomeWorkClassSealed(int i)     {         double f = _mathFunctionClassSealed.Calculate(i);         double dv = _mathFunctionClassSealed.Derivate(i);         return f - (dv * 12.34534);     }      [MethodImpl(MethodImplOptions.NoInlining)]     private static double SomeWorkClass(int i)     {         double f = _mathFunctionClass.Calculate(i);         double dv = _mathFunctionClass.Derivate(i);         return f - (dv * 12.34534);     } } 

The idea of [MethodImpl(MethodImplOptions.NoInlining)] is to prevent the compiler from calculating the addresses of the methods before the loop if the method was inlined.