When running my Express application in production, I want to shut down the server gracefully when its process is killed (i.e. a SIGTERM
or SIGINT
is sent).
Here is a simplified version of my code:
const express = require('express'); const app = express(); app.get('/', (req, res) => res.json({ ping: true })); const server = app.listen(3000, () => console.log('Running…')); setInterval(() => server.getConnections( (err, connections) => console.log(`${connections} connections currently open`) ), 1000); process.on('SIGTERM', shutDown); process.on('SIGINT', shutDown); function shutDown() { console.log('Received kill signal, shutting down gracefully'); server.close(() => { console.log('Closed out remaining connections'); process.exit(0); }); setTimeout(() => { console.error('Could not close connections in time, forcefully shutting down'); process.exit(1); }, 10000); }
When I run it and call the URL http://localhost:3000/ in a browser, the log statement in the setInterval function will keep printing “1 connection currently open” until I actually close the browser window. Even closing the tab will keep the connection open, apparently.
So when I kill my server by hitting Ctrl+C, it will run into the timeout and print “Could not close connections” after 10 seconds, all the while continuing to print “1 connection open”.
Only if I close the browser window before killing the process I get the “closed out remaining connections” message.
What am I missing here? What is the proper way to shut down an Express server gracefully?
To properly close Node Express server, we can call the close method. to call server. close with a callback that has the error err parameter. If err is set, then there's an error when closing the Express server and we call process.
Procedure of Graceful Shutdown: For this purpose, a SIGTERM (the program manager sends it ) signal is sent to the application that tells it that it is going to be killed. After getting this signal, the app stops accepting the new requests, by letting the load balancer know that is not going to accept any new requests.
A graceful shutdown is when a computer is turned off by software function and the operating system (OS) is allowed to perform its tasks of safely shutting down processes and closing connections.
I added a listener for connections opening on the server, storing references to those connections in an array. When the connections are closed, they are removed from the array.
When the server is killed, each of the connection is closed by calling its end
methods. For some browsers (e.g. Chrome), this is not enough, so after a timeout, I call destroy
on each connection.
const express = require('express'); const app = express(); app.get('/', (req, res) => res.json({ ping: true })); const server = app.listen(3000, () => console.log('Running…')); setInterval(() => server.getConnections( (err, connections) => console.log(`${connections} connections currently open`) ), 1000); process.on('SIGTERM', shutDown); process.on('SIGINT', shutDown); let connections = []; server.on('connection', connection => { connections.push(connection); connection.on('close', () => connections = connections.filter(curr => curr !== connection)); }); function shutDown() { console.log('Received kill signal, shutting down gracefully'); server.close(() => { console.log('Closed out remaining connections'); process.exit(0); }); setTimeout(() => { console.error('Could not close connections in time, forcefully shutting down'); process.exit(1); }, 10000); connections.forEach(curr => curr.end()); setTimeout(() => connections.forEach(curr => curr.destroy()), 5000); }
The problem you are experiencing is that all modern browsers reuse single connection for multiple requests. This is called keep-alive connections.
The proper way to handle this is to monitor all new connections and requests and to track status of each connection (is it idle or active right now). Then you can forcefully close all idle connections and make sure to close active connections after current request is being processed.
I've implemented the @moebius/http-graceful-shutdown module specifically designed to gracefully shutdown Express applications and Node servers overall. Sadly nor Express, nor Node itself doesn't have this functionality built-in.
Here's how it can be used with any Express application:
const express = require('express'); const GracefulShutdownManager = require('@moebius/http-graceful-shutdown').GracefulShutdownManager; const app = express(); const server = app.listen(8080); const shutdownManager = new GracefulShutdownManager(server); process.on('SIGTERM', () => { shutdownManager.terminate(() => { console.log('Server is gracefully terminated'); }); });
Feel free to check-out the module, the GitHub page has more details.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With