Why is using a generator function slower than filling and iterating an array in this example?

Question

A Tale of Two Functions

I have one function that fills an array up to a specified value:

function getNumberArray(maxValue) {
    const a = [];

    for (let i = 0; i < maxValue; i++) {
        a.push(i);
    }

    return a;
}

And a similar generator function that instead yields each value:

function* getNumberGenerator(maxValue) {
    for (let i = 0; i < maxValue; i++) {
        yield i;
    }
}

Test Runner

I've written this test for both these scenarios:

function runTest(testName, numIterations, funcToTest) {
    console.log(`Running ${testName}...`);
    let dummyCalculation;
    const startTime = Date.now();
    const initialMemory = process.memoryUsage();
    const iterator = funcToTest(numIterations);

    for (let val of iterator) {
        dummyCalculation = numIterations - val;
    }

    const finalMemory = process.memoryUsage();

    // note: formatNumbers can be found here: https://jsfiddle.net/onz1ozjq/
    console.log(formatNumbers `Total time: ${Date.now() - startTime}ms`);
    console.log(formatNumbers `Rss:        ${finalMemory.rss - initialMemory.rss}`);
    console.log(formatNumbers `Heap Total: ${finalMemory.heapTotal - initialMemory.heapTotal}`);
    console.log(formatNumbers `Heap Used:  ${finalMemory.heapUsed - initialMemory.heapUsed}`);
}

Running the Tests

Then when running these two like so:

const numIterations = 999999; // 999,999
console.log(formatNumbers `Running tests with ${numIterations} iterations...\n`);
runTest("Array test", numIterations, getNumberArray);
console.log("");
runTest("Generator test", numIterations, getNumberGenerator);

I get results similar to this:

Running tests with 999,999 iterations...

Running Array test...
Total time: 105ms
Rss:        31,645,696
Heap Total: 31,386,624
Heap Used:  27,774,632

Running Function generator test...
Total time: 160ms
Rss:        2,818,048
Heap Total: 0
Heap Used:  1,836,616

Note: I am running these tests on node v4.1.1 on Windows 8.1. I am not using a transpiler and I'm running it by doing node --harmony generator-test.js.

Question

The increased memory usage with an array is obviously expected... but why am I consistently getting faster results for an array? What's causing the slowdown here? Is doing a yield just an expensive operation? Or maybe there's something up with the method I'm doing to check this?

Answer 1

The terribly unsatisfying answer is probably this: Your ES5 function relies on features that (with the exceptions of let and const) have been in V8 since it was released in 2008 (and presumably for some time before, as I understand that what became V8 originated as part of Google's web crawler). Generators, on the other hand, have only been in V8 since 2013. So not only has the ES5 code had seven years to be optimized while the ES6 code has had only two, almost nobody (compared to the many millions of sites using code just like your ES5 code) is using generators in V8 yet, which means there has been very little opportunity to discover, or incentive to implement, optimizations for it.

If you really want a technical answer as to why generators are comparatively slow in Node.js, you'll probably have to dive into the V8 source yourself, or ask the people who wrote it.

Answer 2

FYI this question is ancient in internet terms and generators have caught up (at least when tested in Chrome) https://jsperf.com/generator-vs-loops1