How many concurrent setTimeouts before performance issues?

Question

I have a node.js app with 10k-100k concurrent setTimeouts running at any given time. (They are all 5 minute duration.) The callback is pretty trivial, just an HDECRBY in redis. I have not hit any performance issues yet with this, even on a t2.micro instance.

I know that I will run into issues if the callback functions can't get executed as fast as I'm setting the setTimeouts (obviously), but are there issues with having a high number of setTimeouts, per se? e.g., am I going to run in to a RAM bottleneck if I scale this up to, say, 1 million concurrent? 10 million?

Answer 1

For these types of questions, it is often useful to just go look at how node.js handles timers in the source code.

What you will find is that node.js keeps one or more linked lists of its own internal timer objects and all timers set to occur at the same time share one libuv timer. This means that zillions of timers all set to occur in a fairly specific time window will inevitably share many firing times and thus will share timer lists and thus will share many system timer objects.

This makes it less of an issue for having zillions of timer objects. Now, every timer object still requires some memory and not every operation in the timer implementation is of constant time though you can see in the comments below, they tried to make as many of them as possible to be constant time to allow large numbers of timers with still decent performance.

If you don't need absolute precision in exactly when the timer fires, you can probably cause your timers to coalesce and share timer objects more often by scheduling your timers only for specific time boundaries such as an even number of 100ms. This would be scheduling more of your zillions of timers for the same firing time and would allow node.js to put more timers into the same list that all share a single system timer. I don't know whether this is feasible with your timers or if it is even needed, but in studying how node.js works, it would increase efficiency. There would be both fewer timer lists internally to node.js and fewer system timers in libuv.

---

Here are some explanatory comments from the node.js code on timers that explains some more aspects of the design:

// HOW and WHY the timers implementation works the way it does.
//
// Timers are crucial to Node.js. Internally, any TCP I/O connection creates a
// timer so that we can time out of connections. Additionally, many user
// user libraries and applications also use timers. As such there may be a
// significantly large amount of timeouts scheduled at any given time.
// Therefore, it is very important that the timers implementation is performant
// and efficient.
//
// Note: It is suggested you first read though the lib/internal/linkedlist.js
// linked list implementation, since timers depend on it extensively. It can be
// somewhat counter-intuitive at first, as it is not actually a class. Instead,
// it is a set of helpers that operate on an existing object.
//
// In order to be as performant as possible, the architecture and data
// structures are designed so that they are optimized to handle the following
// use cases as efficiently as possible:

// - Adding a new timer. (insert)
// - Removing an existing timer. (remove)
// - Handling a timer timing out. (timeout)
//
// Whenever possible, the implementation tries to make the complexity of these
// operations as close to constant-time as possible.
// (So that performance is not impacted by the number of scheduled timers.)
//
// Object maps are kept which contain linked lists keyed by their duration in
// milliseconds.
// The linked lists within also have some meta-properties, one of which is a
// TimerWrap C++ handle, which makes the call after the duration to process the
// list it is attached to.
//
//
// ╔════ > Object Map
// ║
// ╠══
// ║ refedLists: { '40': { }, '320': { etc } } (keys of millisecond duration)
// ╚══          ┌─────────┘
//              │
// ╔══          │
// ║ TimersList { _idleNext: { }, _idlePrev: (self), _timer: (TimerWrap) }
// ║         ┌────────────────┘
// ║    ╔══  │                              ^
// ║    ║    { _idleNext: { },  _idlePrev: { }, _onTimeout: (callback) }
// ║    ║      ┌───────────┘
// ║    ║      │                                  ^
// ║    ║      { _idleNext: { etc },  _idlePrev: { }, _onTimeout: (callback) }
// ╠══  ╠══
// ║    ║
// ║    ╚════ >  Actual JavaScript timeouts
// ║
// ╚════ > Linked List
//
//
// With this, virtually constant-time insertion (append), removal, and timeout
// is possible in the JavaScript layer. Any one list of timers is able to be
// sorted by just appending to it because all timers within share the same
// duration. Therefore, any timer added later will always have been scheduled to
// timeout later, thus only needing to be appended.
// Removal from an object-property linked list is also virtually constant-time
// as can be seen in the lib/internal/linkedlist.js implementation.
// Timeouts only need to process any timers due to currently timeout, which will
// always be at the beginning of the list for reasons stated above. Any timers
// after the first one encountered that does not yet need to timeout will also
// always be due to timeout at a later time.
//
// Less-than constant time operations are thus contained in two places:
// TimerWrap's backing libuv timers implementation (a performant heap-based
// queue), and the object map lookup of a specific list by the duration of
// timers within (or creation of a new list).
// However, these operations combined have shown to be trivial in comparison to
// other alternative timers architectures.


// Object maps containing linked lists of timers, keyed and sorted by their
// duration in milliseconds.
//
// The difference between these two objects is that the former contains timers
// that will keep the process open if they are the only thing left, while the
// latter will not.