Node child processes: how to intercept signals like SIGINT

Question

In my Node app, I'm hooking on the SIGINT signal in order to gracefully stop (using pm2, but this is not relevant here).

My app also execs/spawns a couple of child processes.

I am able to hook on SIGINT to intercept it and perform graceful stop, however my child processes are passed through the same signal, and thus, instantly killed.

How can I intercept the SIGINT signal on my child processes?

A sample of what I'm doing:

const child = child_process.spawn('sleep', ['10000000']);
console.log(`Child pid: ${child.pid}`);

child.on('exit', (code, signal) => { console.log('Exit', code, signal); });

process.on('SIGINT', () => {
    console.log("Intercepting SIGINT");
});

Answer 1

By default, child processes created by child_process.spawn() have the same process group as the parent, unless they were called with the {detached:true} option.

The upshot is that this script will behave differently in different environments:

// spawn-test.js
const { spawn } = require('child_process');
const one = spawn('sleep', ['101']);
const two = spawn('sleep', ['102'], {detached: true});
two.unref();
process.on('SIGINT', function () {
  console.log('just ignore SIGINT');
});

On interactive shells, a SIGINT from Ctl-C is sent to the whole group by default, so the non-detached child will get the SIGINT and exit:

you@bash $ node spawn-test.js
^Cjust ignore SIGINT
# the parent process continues here, let's check children in another window:
you@bash [another-terminal-window] $ ps aux | grep sleep
... sleep 102
# note that sleep 101 is not running anymore
# because it recieved the SIGINT from the Ctl-C on parent

...but calls to kill(2) can just signal your parent process, so children stay alive:

you@bash $ node spawn-test.js & echo $?
[2] 1234
you@bash [another-terminal-window] $ kill -SIGINT 1234
you@bash [another-terminal-window] $ ps aux | grep sleep
... sleep 101
... sleep 102
# both are still running

However, pm2 is a whole other beast. Even if you try the above techniques, it kills the whole process tree, including your detached process, even with a long --kill-timeout:

# Test pm2 stop
you@bash $ pm2 start spawn-test.js --kill-timeout 3600
you@bash $ pm2 stop spawn-test
you@bash $ ps aux | grep sleep
# both are dead

# Test pm3 reload
you@bash $ pm2 start spawn-test.js --kill-timeout 3600
you@bash $ pm2 reload spawn-test
you@bash $ ps aux | grep sleep
# both have different PIDs and were therefore killed and restarted

This seems like a bug in pm2.

I've gotten around similar problems by using the init system (systemd in my case) rather than pm2, since this allows for greater control over signal handling.

On systemd, signals are sent to the whole group by default, but you can use KillMode=mixed to have the signal sent to the parent process only, but still SIGKILL child processes if they run beyond the timeout.

My systemd unit files look like this:

[Unit]
Description=node server with long-running children example

[Service]
Type=simple
Restart=always
RestartSec=30
TimeoutStopSec=3600
KillMode=mixed
ExecStart=/usr/local/bin/node /path/to/your/server.js

[Install]
WantedBy=multi-user.target

Answer 2

Normally in C, you'd solve this by ignoring the signal in the child (or by spawning it in a new process group so that the terminal generated signal for the foreground process group doesn't reach it).

From looking at https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options, it doesn't look like NodeJs exposes an API for this, however, it does have an option for spawning the child process through the shell, so what you can do is turn it on and ignore the signal in the shell, which will cause its ignored status to be inherited to the shell's children.

const child_process = require('child_process')
//const child = child_process.spawn('sleep', ['10000000']);
const child = child_process.spawn("trap '' INT; sleep 10000000", [], {shell: true });
console.log(`Child pid: ${child.pid}`);

child.on('exit', (code, signal) => { console.log('Exit', code, signal); });

process.on('SIGINT', () => {
    console.log("Intercepting SIGINT");
});


//emulate cat to keep the process alive
process.stdin.pipe(process.stdout);

Now when you press Ctrl-C, the Node process handles it and the sleep process lives on. (In case you're unfamiliar with the other terminal generated signals, you can easily kill this group by pressing Ctrl-\ (sends SIGQUIT to the group) if you don't mind the coredump).