I am using the NodeJS VM Module to run untrusted code safely. I have noticed a huge memory leak that takes about 10M of memory on each execution and does not release it. Eventually, my node process ends up using 500M+ of memory. After some digging, I traced the problem to the constant creation of VMs. To test my theory, I commented out the code that creates the VMs. Sure enough, the memory usage dropped dramatically. I then uncommented the code again and placed global.gc() calls strategically around the problem areas and ran node with the--expose-gc flag. This reduced my memory usage dramatically and retained the functionality.
Is there a better way of cleaning up VMs after I am done using it?
My next approach is the cache the vm containing the given unsafe code and reusing it if it I see the unsafe code again (Background:I am letting users write their own parsing function for blocks of text, thus, the unsafe code be executed frequently or executed once and never seen again).
Some reference code.
async.each(items,function(i,cb){
// Initialize context...
var context = vm.createContext(init);
// Execute untrusted code
var captured = vm.runInContext(parse, context);
// This dramatically improves the usage, but isn't
// part of the standard API
// global.gc();
// Return Result via a callback
cb(null,captured);
});
Avoid Accidental Globals This could be the result of a typo and could lead to a memory leak. Another way could be when assigning a variable to this within a function in the global scope. To avoid issues like this, always write JavaScript in strict mode using the 'use strict'; annotation at the top of your JS file.
How To Detect. There're many tools and libraries used to detect memory leaks in NodeJS, all following the same concept to detect memory leaks by comparing different heap dumps and check the results, and they try to force run the garbage collector before taking any heap dump to make sure that the leakage is real.
The Memory Heap is divided into two major spaces: Old space: where older objects are stored. Usually, objects are moved here after surviving in a new space for some time. The old space can be controlled by the flag --max-old-space-size. New space: most objects are allocated here.
When I see this right this was fixed in v5.9.0
, see this PR. It appears that in those cases both node
core maintainer nor programmers can do much - that we pretty much have to wait for a upstream fix in v8
.
So no, you can't do anything more about it. Catching this bug was good though!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With