Update nodejs in production with zero downtime

Question

There are several packages that can update node js app in production mode with zero downtime (or graceful reload), for example pm2.

However, is it possible to update node js itself, for example from the LTS version 4.x to the new version 6.x in production with zero downtime?

Answer 1

Updating production with zero downtime can be done with whatever you want to update as long as it has redundancy. We do it every day at work.

Upgrading node is routine as long as installing node is part of your deployment process, using for example nvm

You need several servers of course.

Prerequisite : Suppose your code is version 1.0, upgrade node.js in dev, test your code, persist the required node version (package.json, .nvmrc or whatever your install script need) and bump it to 1.1 Extra check that your server OS has node 6 requirements. (for ex. Centos 6 can't install node 4)

Generic zero-downtime rolling-deploy procedure, supposing you have 4 servers in a farm :

1. remove the server 1 from the farm. If you use persistent connections (websocket), signal existing clients of this server to reconnect (to servers 2, 3 and 4)
2. deploy version 1.1 to the new server. This should include the reinstallation of node (ex. nvm install) Connect directly to it and check everything is ok.
3. do the same with server 2 (remove from farm, signal clients, deploy new version) so we don't have a single point of failure. Your app is still being served by servers 3 and 4
4. Put back servers 1 and 2 in the farm and remove 3 and 4. Signal 3/4 clients to reconnect if needed.
5. upgrade servers 3 and 4 and put them back in the farm

Done. All the servers are upgraded and the clients didn't notice anything if your app is well coded.

Answer 2

To do zero-downtime you need to make sure someone is handling user requests all the time, so at the exact moment you kill the old version's process, new one has to already be bound to the same port with the same app.

So the most important part of zero-downtime in this case is load balancing. For that, you should have a HAproxy, nginx etc. installed in front of your app with multiple node processes, so that you can kill and replace each one separately.

In general, running one node process for all your production is not a good idea, so if you don't have any load balancer do install one even at a cost of some downtime. Then put at least two separate servers (or containers) behind it and run your app with pm2 on it.

Also, notice that even with redundancy your app will interrupt clients if you just restart one node - eg. if someone is uploading a file while this happens.

HAproxy has a feature where you can disable one of the backends, but wait till all the http connections to it finish in a natural way while haproxy is not sending any new traffic to it. That's how you make sure no user is interrupted.

Example configuration here: https://blog.deimos.fr/2016/01/21/zero-downtime-upgrade-with-ansible-and-haproxy/