Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kill all Node cluster workers once operation is complete

I'm trying to create a cluster to spread hashing between CPUs, the one who finds the hash return that for me to use and kill all other workers when the first one responds.

I started my code by just creating the clusters and running the function, then added the "send" back to the master and then tried adding the logic to kill all workers,

I've read the documentation Killing node.js workers after function is done as reference but it doesn't seem to work - I can see a Node operation still running in the background (on a 2-core machine) using a ton of CPU, then I'll get some console log even when the Node process is finished and back on the bash terminal.

I can't for the life of me figure out where I'm going wrong so any assistance would be appreciated.

My current code is:

if (cluster.isMaster) {

    for (let i = 0; i < numCPUs; i++) {
        let worker = cluster.fork();
    }
    cluster.on('exit', function(worker, code, signal) {
        for (var id in cluster.workers) {
            cluster.workers[id].kill();
        }
        process.exit(0);
    });

    function messageHandler(msg) {
        console.log(msg);
        if (msg.hash.length > 1) {
            console.log(msg.hash);
        }
    }

    for (const id in cluster.workers) {
        cluster.workers[id].on('message', messageHandler);
    }

} else {

    console.log(`Worker ${process.pid} started and finished`)
    console.log(parseInt(cluster.worker.id));
    let difficulty = 5;
    i = cluster.worker.id;
    var start = new Date();
    var hrstart = process.hrtime();
    hash = computeHash(index, lasthash, timestamp, data, i);

    while (hash.substring(0, difficulty) !== Array(difficulty + 1).join("0")) {
        hash = computeHash(index, lasthash, timestamp, data, i);
        i = i + cluster.worker.id;
    }

    var end = new Date() - start,
        hrend = process.hrtime(hrstart);
    console.info('Execution time (hr): %ds %dms', hrend[0], hrend[1] / 1000000)
    console.log("Hash found from: " + i);
    process.send({
        hash: hash
    });
    process.exit(0);
}
like image 475
orgg Avatar asked Nov 06 '22 03:11

orgg


1 Answers

Okay so I managed to fix this in quite a hacky way. I discovered on Windows that the answer provided elsewhere does work (ie):

for (var id in cluster.workers) {
    cluster.workers[id].kill();
 }

However on Linux, you'll still have the process running even if you terminate the master. If you're like me and using blocking code on your child, it won't work (I also couldn't add a timeout into the function, for whatever reason).

The way I fixed this was to get a list of all remaining workers and their pids inside the loop, as before, only this time use process.kill with the pid of the remaining workers (like so:)

  for (var id in cluster.workers) {
   console.log("Killing remaining processes");
   let process_id = cluster.workers[id].process.pid;
   process.kill(process_id);
 }

This solution is hacky, but it works and there is very limited examples out there so I hope this can help someone.

like image 192
orgg Avatar answered Nov 15 '22 07:11

orgg