Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using cluster in an Expressjs app

I'm doing a little OJT on my first node project and, while I can stand up a simple server, the app is going to get hammered so using cluster seems like a good idea. I've cobbled together some code snippets that I've found in various searches (including SO), but the server won't start. I'm sure my inexperience with node has me doing something stupid, but I don't see it.

var express = require( 'express' );
var cluster = require( 'cluster' );
var path    = require( 'path' );

var cCPUs   = require( 'os' ).cpus().length;
var port    = 3000;
var root    = path.dirname( __dirname );

if( cluster.isMaster ) {
    for( var i = 0; i < cCPUs; i++ ) {
      cluster.fork();
    }

    cluster.on( 'death', function( worker ) {
      console.log( 'Worker ' + worker.pid + ' died.' );
    });
}
else {
    // eyes.inspect( process.env );
    console.log( 'Worker: %s', process.env.NODE_WORKER_ID );

    var app = express();
    var routes  = require( './routes' )( app );
    app
      .use( cluster.repl( root + 'cluster.repl' ) )
      .use( cluster.stats({ connections: true, requests: true }) )
      .use( cluster.reload( root ) )
      .listen( port );
}

RESULT:

TypeError: Object #<Cluster> has no method 'repl'

If I remove the use calls, the workers start up correctly, but process.env.NODE_WORKER_ID is undefined. Inspecting process.env shows me that it's definitely not defined. Maybe the snippet I used was from an old version, but I'm not sure how to identify the worker thread in any other way.

If anyone can unscrambled whatever I've scrambled, I'd really appreciate it.

like image 290
Rob Wilkerson Avatar asked Jan 18 '13 18:01

Rob Wilkerson


People also ask

When would you use cluster module in node JS?

Definition and Usage Node. js runs single threaded programming, which is very memory efficient, but to take advantage of computers multi-core systems, the Cluster module allows you to easily create child processes that each runs on their own single thread, to handle the load.

Is ExpressJS outdated?

Express has not been updated for years, and its next version has been in alpha for 6 years. People may think it is not updated because the API is stable and does not need change. The reality is: Express does not know how to handle async/await .

Is it possible to cluster multiple node processes?

Node. js applications can be parallelized using cluster modules in order to use the system more efficiently. Running multiple processes at the same time can be done using few lines of code and this makes the migration relatively easy, as Node.

What is cluster mode in NodeJS?

A Look at Clustering The Node. js Cluster module enables the creation of child processes (workers) that run simultaneously and share the same server port. Each spawned child has its own event loop, memory, and V8 instance. The child processes use IPC (Inter-process communication) to communicate with the parent Node.


2 Answers

For anyone searching later, here's what I ended up with:

const cluster = require('cluster');
const express = require('express');
const path = require('path');

const port = 3000;
const root = path.dirname(__dirname);
const cCPUs = require('os').cpus().length;

if (cluster.isMaster) {
    // Create a worker for each CPU
    for (let i = 0; i < cCPUs; i++) {
        cluster.fork();
    }
    cluster.on('online', function (worker) {
        console.log('Worker ' + worker.process.pid + ' is online.');
    });
    cluster.on('exit', function (worker, code, signal) {
        console.log('worker ' + worker.process.pid + ' died.');
    });
} else {
    const app = express();
    const routes = require('./routes')(app);
    app.use(express.bodyParser()).listen(port);
}

I'm still very early in the node learning curve, but the server starts and appears to have a working running on each core. Thanks to JohnnyH for getting me on the right track.

like image 113
Rob Wilkerson Avatar answered Sep 18 '22 07:09

Rob Wilkerson


Also take a look at cluster2. It's used by eBay and has an express example

var Cluster = require('cluster2'),
    express = require('express');

var app = express.createServer();

app.get('/', function(req, res) {
  res.send('hello');
});

var c = new Cluster({
  port: 3000,
});

c.listen(function(cb) {
  cb(app);
});
like image 36
zemirco Avatar answered Sep 19 '22 07:09

zemirco