How to increase EsLint memory to avoid `JavaScript heap out of memory`?

Question

I'm trying to run EsLint on a very large javascript file and the process is running out of memory. To give you an idea of how big the file is I ran Cloc on it and here's the output:

$ cloc app.js 
       1 text file.
       1 unique file.                              
       0 files ignored.

github.com/AlDanial/cloc v 1.80  T=12.81 s (0.1 files/s, 42499.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                       1           4255          23744         516524
-------------------------------------------------------------------------------

The file weighs 23MB.

$ ls -lAh app.js 
-rw-r--r-- 1 miguelangel staff 23M Jan 28 11:58 app.js

This is all to say this is very probably not a memory leak in EsLint. I've seen some issues with EsLint having memory leaks in Github.com. I don't think that's the case here.

The file is this big because it's the result of concatenating many other Javascript modules. My aim here is to try to find any unused code. The code-base of this project has obviously grown without control and I'm trying to skim off dead weight. So I'm trying to run EsLint's no-unused-vars rule on a concatenation of the whole code-base. Here's my EsLint config file:

.eslintrc.js

module.exports = {
    "env": {
        "browser": true,
        "commonjs": false,
        "es6": true
    },
    "parserOptions": {
        "ecmaVersion": 2015
    },
    "rules": {
        "no-unused-vars": [
            "warn"
        ]
    }
};

This project is not a Node nor AMD project so I figured I had to put the entire code-base in a single file to avoid false positives.

The problem is that trying to run EsLint on this file results in a JavaScript heap out of memory error.

$ eslint app.js 

<--- Last few GCs --->

[60451:0x104002200]    43814 ms: Mark-sweep 1395.7 (1424.1) -> 1395.2 (1423.6) MB, 5719.6 / 0.0 ms  (+ 0.1 ms in 28 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 5755 ms) (average mu = 0.148, current mu = 0.037) alloca[60451:0x104002200]    49447 ms: Mark-sweep 1397.4 (1424.1) -> 1396.9 (1425.6) MB, 5569.8 / 0.0 ms  (+ 0.1 ms in 11 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 5598 ms) (average mu = 0.081, current mu = 0.011) alloca

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3275f3d4fb7d]
Security context: 0x14c691f9d969 <JSObject>
    1: /* anonymous */ [0x14c6f1b7b981] [/usr/local/lib/node_modules/eslint/node_modules/acorn/dist/acorn.js:~2868] [pc=0x3275f40f5843](this=0x14c6b794c669 <Parser map = 0x14c603088f11>)
    2: /* anonymous */ [0x14c6f1b7b111] [/usr/local/lib/node_modules/eslint/node_modules/acorn/dist/acorn.js:2190] [bytecode=0x14c691fecb01 offset=968](this=0x14c6b794c669 <...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003ace0 node::Abort() [/usr/local/bin/node]
 2: 0x10003aeb1 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x10018c8cf v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10018c870 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x10047b188 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x10047cc01 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x10047a4c4 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x100479236 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x100481826 v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x100481b5c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
11: 0x100461562 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
12: 0x100653464 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x3275f3d4fb7d 
Abort trap: 6

How can I increase EsLint's access to memory?

Answer 1

I've browsed the EsLint docs hoping to find an option to configure it to run with more memory. Sadly I was not able to find anything of the sort.

However, inspired by @vsemozhetbyt's answer -hence the up-vote- I started looking at Node's configuration options and I've found a way to get around this using Node's NODE_OPTIONS environment variable.

A space-separated list of command line options. options... are interpreted as if they had been specified on the command line before the actual command line

So this is what I did:

$ export NODE_OPTIONS="--max-old-space-size=4096"
$ echo $NODE_OPTIONS
--max-old-space-size=4096
$ eslint app.js

This way the Node process that runs EsLint just picks this up without the need for me to type in the --max-old-space-size flag nor the path to the Node binary every time I invoke EsLint.

Answer 2

By default, Node.js sets memory limit near 1.5 GB. You can increase it by --max_old_space_size key (say, --max_old_space_size=4096 to use 4 GB).

To use the key, you need to know the path to the ESLint main file and call it as a script this way:

node --max_old_space_size=4096 ./node_modules/eslint/bin/eslint.js app.js

Answer 3

I was having this problem in a Laravel/Vue application. Those details aren't significant, but the fact that Laravel has some folders such as vendor that includes all the composer packages meant that ES Lint was traversing a huge number of files, enough to blow out the heap.

The solution was to first add a .eslintignore file into the root of the project, such as:

/app
/bootstrap
/config
/database
/node_modules
/public
/routes
/storage
/tests
/vendor

Next, to execute ES Lint from the CLI (and GitLab-CI) as:

node_modules/eslint/bin/eslint.js --config ./.eslintrc.json .

It was also important to make sure ES Lint was using the correct config file to check against the intended ruleset.

You can visually test against some of this by adding --debug onto your CLI command, and you will see it traversing illegal folders and files if that is occurring. I think that is a decent debug step--to test it in verbose mode. You might see ES Lint traversing unnecessary folders or testing against incorrect rules.

In my opinion, this problem can easily stem from ES Lint trying to load a huge number of errors into memory while it is working. Rather than increase the heap, you can dial in what exactly ES Lint is linting.