Node.js memory filling up

Question

I'm trying to write a program that downloads a large zip file, unzips it in memory, and then pushes the contents (a series of CSV files) to MongoDB. However, I keep hitting a point where the program halts and prints

FATAL ERROR: CodeRange::GetNextAllocationBlock Allocation failed - process out of memory

I've been setting buffers to null when they're no longer in use, setting records to null once they're in Mongo, and preventing more than one file from processing at a time. None of this has helped. Are there any more tricks to releasing memory?

Answer 1

Somethings I would consider would be(not sure if it will all work out as desired):

Make sure references are gone:

• https://groups.google.com/d/msg/nodejs/ddXqNI_e_pU/NKMHmz4RLsoJ

Performing GC manually and increase v8 heap size:

• http://blog.caustik.com/2012/04/11/escape-the-1-4gb-v8-heap-limit-in-node-js/

Spawning(and killing) child process from parent process to do work:

• http://www.robsearles.com/2011/09/28/nodejs-experiments-with-processes/

That way I think the OS will reclaim memory for child process even if it is not returning memory.

Ram as filesystem:

• http://www.cyberciti.biz/faq/howto-create-linux-ram-disk-filesystem/

That way you can treat filesystem as memory(let operating system put it into memory).

Use freelist:

• https://groups.google.com/d/msg/nodejs/ddXqNI_e_pU/EP0xUi04xl8J