Java Reflection Performance Issue

Question

I know there's a lot of topics talking about Reflection performance.

Even official Java docs says that Reflection is slower, but I have this code:

  public class ReflectionTest {
   public static void main(String[] args) throws Exception {
       Object object = new Object();
       Class<Object> c = Object.class;

       int loops = 100000;

       long start = System.currentTimeMillis();
       Object s;
       for (int i = 0; i < loops; i++) {
           s = object.toString();
           System.out.println(s);
       }
       long regularCalls = System.currentTimeMillis() - start;
       java.lang.reflect.Method method = c.getMethod("toString");

       start = System.currentTimeMillis();
       for (int i = 0; i < loops; i++) {
           s = method.invoke(object);
           System.out.println(s);
       }

       long reflectiveCalls = System.currentTimeMillis() - start;

       start = System.currentTimeMillis();
       for (int i = 0; i < loops; i++) {
           method = c.getMethod("toString");
           s = method.invoke(object);
           System.out.println(s);
       }

       long reflectiveLookup = System.currentTimeMillis() - start;

       System.out.println(loops + " regular method calls:" + regularCalls
               + " milliseconds.");

       System.out.println(loops + " reflective method calls without lookup:"
               + reflectiveCalls+ " milliseconds.");

       System.out.println(loops + " reflective method calls with lookup:"
               + reflectiveLookup + " milliseconds.");

   }

}

That I don't think is a valid benchmark, but at least should show some difference. I executed it waiting to see the reflection normal calls being a bit slower than regular ones.

But this prints this:

100000 regular method calls:1129 milliseconds.
100000 reflective method calls without lookup:910 milliseconds.
100000 reflective method calls with lookup:994 milliseconds.

Just for note, first I executed it without that bunch of sysouts, and then I realized that some JVM optimization are just making it goes faster, so I added these printls to see if reflection was still faster.

The result without sysouts are:

100000 regular method calls:68 milliseconds.
100000 reflective method calls without lookup:48 milliseconds.
100000 reflective method calls with lookup:168 milliseconds.

I saw over internet that the same test executed on old JVMs make the reflective without lookup are two times slower than regular calls, and that speed falls over new updates. If anyone can execute it and say me I'm wrong, or at least show me if there's something different than the past that make it faster.

Following instructions, I ran every loop separated and the result are (without sysouts)

100000 regular method calls:70 milliseconds.
100000 reflective method calls without lookup:120 milliseconds.
100000 reflective method calls with lookup:129 milliseconds.

Answer 1

Never performance test different bits of code in the same "run". The JVM has various optimisations that mean it though the end result is the same, how the internals are performed may differ. In more concrete terms, during your test the JVM may have noticed you are calling Object.toString a lot and have started to inline the method calls to Object.toString. It may have started to perform loop unfolding. Or there could have been a garbage collection in the first loop but not the second or third loops.

To get a more meaningful, but still not totally accurate picture you should separate your test into three separate programs.

The results on my computer (with no printing and 1,000,000 runs each)

All three loops run in same program

1000000 regular method calls: 490 milliseconds.

1000000 reflective method calls without lookup: 393 milliseconds.

1000000 reflective method calls with loopup: 978 milliseconds.

Loops run in separate programs

1000000 regular method calls: 475 milliseconds.

1000000 reflective method calls without lookup: 555 milliseconds.

1000000 reflective method calls with loopup: 1160 milliseconds.

Answer 2

There's an article by Brian Goetz on microbenchmarks that's worth reading. It looks like you're not doing anything to warm up the JVM (meaning give it a chance to do whatever inlining or other optimizations it's going to do) before doing your measurements, so it's likely the non-reflective test is still not warmed-up yet, and that could skew your numbers.