Why does running this loop 9 times take 100x longer than running it 8 times?

Question

Consider this code:

Test = function() {
}

t = new Test();

for (var i = 0; i < 8; i++) {
  result = t instanceof Test;
}

If you change the number of iterations from 8 to 9, the loop will suddenly take about 100 times longer to complete in the latest version of Firefox (41.0.1). I tested this on two different PCs and the magic limit is always 8.

Here is the JSPerf test that I used: http://jsperf.com/instanceof-8-times-vs-9-times

Does anyone have any idea why that might happen? It seems to be specific to instanceof. It does not happen if you do something else with the object, for example check a property.

---

Note: I also filed a Bugzilla bug about this.

Answer 1

Jan de Mooij from the Mozilla team has posted some details in the Bugzilla thread. Here's my simplistic interpretation of his highly technical answers:

In the i < 8 case, Firefox is smart enough to hoist the result = t instanceof Test; statement out of the loop (according to my tests, it doesn't seem to omit it altogether). In the i < 9 case, it apparently doesn't do that optimization.

Why? The reason is not exactly clear, but it is probably related to the fact that 9 iterations is the threshold above which the function is considered "hot" enough to run it through the JIT compiler. The i < 8 case stays in the interpreter. (I don't see why JIT-ing would exclude hoisting, but apparently it does in the current version of the engine.)

Interestingly, the 8-iteration threshold doesn't seem to be universal. For example, if we replace our own prototype (Test) with a built-in prototype (e.g. CustomEvent), hoisting doesn't seem to occur regardless of the number of iterations (relevant JSPerf):

for (var i = 0; i < 8; i++) { //or i < 9
  t instanceof CustomEvent;
}

Coming back to the original code using the Test prototype, why is performance so bad in the i < 9 case? This is related to how JSPerf works. The "setup" code is not just executed once – it is run once "per test". Every time you click Run, JSPerf runs hundreds of "tests", each test comprising thousands of iterations. So the setup code is run hundreds of times. This means that there are hundreds of different prototype objects named Test in the program, all created with the line:

Test = function(){
}

The Ion JIT optimizing compiler can easily optimize the case in which we are using instanceof on the same prototype object many times (as we do with CustomEvent in this test case), but when it notices that there is more than one object with the same name, apparently it throws its hands in the air.

Jan has rightly pointed out that this is not likely to affect too many real-world scripts because normally a single identifier is associated with a single prototype object (e.g. you have a class Foobar, which is defined only once and never re-defined). But JSPerf re-defines prototypes hundreds of times. It seems to me that this fact casts serious doubt on all published JSPerf results which include prototype definitions, except those which explicitly avoid redefinition by using globals (as in this test case) – which is perhaps the most important conclusion from all this.

For example, the JSPerf tests linked from this question: Is using instanceof operator in javascript a performance issue? are probably worthless, since they all define prototypes in the setup code.