How to properly scale nodejs app on heroku using clusters

Question

I noticed a warning in heroku logs lately saying that seemed new to me

web.1: Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)

web.1: Recommending WEB_CONCURRENCY=1

I did some research and found this cluster article, which is the "default" way to use clusters in nodejs, but it totally contradicts with a newly updated article that contains this new WEB_CONCURRENCY environment variable and with a different size suggestion for each dyno (which is much smaller, btw)

Answer 1

The first link is from July 2014, and used to be the recommended way of doing things. However, Heroku's dynos are quite memory-centric, and it is very easy to exceed the allocated memory allowance when using the maximum number of cores per CPU (as the first article suggests).

Instead, the new recommendation is to profile your app and figure out how much memory is required per process. Set an environment variable WEB_MEMORY to this value, and then update your cluster code to the following:

var cluster = require('cluster');
var numWorkers = process.env.WEB_CONCURRENCY;

if(cluster.isMaster) {
  // Master process: fork our child processes
  for (var i = 0; i < numWorkers; i++) {
    cluster.fork();
  }

  // Respawn any child processes that die
  cluster.on('exit', function() {
    cluster.fork();
  });

} else {
  // Child process, put app initialisation code here.
}

By using the WEB_MEMORY variable, Heroku can generate a WEB_CONCURRENCY value depending on the size of the dyno you are running, and hence fork the correct number of processes to ensure that your app doesn't exceed memory allowance.

As an aside, if you do exceed the memory allocation (512MB per dyno for a 1x dyno), swap space will be used for the excess. This will slow down your app, causing request times to increase and will generally contribute to sluggishness. If you exceed the memory usage by too much (approx three times the allocation), Heroku will restart your dyno.