nodeJs huge array processing throws RangeError: Maximum call stack size exceeded

Question

This is part of code to proceed big number of entries ( originally its works with file system and make some operations with files) . Is there any nice way to bypass the limitation and prevent throwing of RangeError: Maximum call stack size exceeded ( As for now it allows me to iterate about 3000 items )

var async = require('async')
    , _u = require('underscore')

var tifPreview = function (item, callback) {
    console.log(item)
    return callback();
}

var tifQueue = async.queue(tifPreview, 2)

tifQueue.push(_u.range(0, 5000, 1))

Answer 1

The problem is that you are making to many function calls. Setting the stack-size to a higher value will only increase the number of items you can handle, not solve the actual problem.

You are calling the next iteration straight from your function, which makes it a recursive function. It's a bit hard to spot since it's going thru async.

This code should work:

var tifPreview = function (item, callback) {
  console.log(item);

  // defer the callback
  setImmediate(callback);
}

Read more about the setImmediate function here: http://nodejs.org/api/timers.html#timers_setimmediate_callback_arg

Answer 2

An option could be passing --max-stack-size to node.

node --max-stack-size 32000 app.js

For reference, use node -h

--max-stack-size=val set max v8 stack size (bytes)

Update

Even though help prints it as --max-stack-size, in node -v v0.10.x+ you need to use --stack-size instead.

node --stack-size=32000 app.js