Is Node using 100% CPU?

Question

I'm a little confused with what I'm seeing with a node process that I have running. docker stats on the host is showing that the container is using over 100% CPU. This makes me think that the node process is maxing out the CPU. This is confirmed when I run top on the host and see that the node process is using over 100% CPU.

When I jump into the docker container I see that node is only using 54% of the CPU and that the processing is split between the two cores. I was expecting to see one core maxed out and the other at 0 since Node is single-threaded.

I found this QA and it looks like the OS could be moving the process between the cores (news to me). Is This Single Node.JS App Using Multiple Cores?

Can you help me interpret the results? Is node pretty much maxed out? Or since the process in the container is showing 54% usage can that go up to 100%? Why is the top in the node container showing 54% usage for node but 45% + 46% for both cores. Nothing is running in the container but the single node process. I'm not using clustering although maybe a package I have included is.

I'm asking all this as I'm trying to understand if I should be scaling this ECS instance out or if node can handle more.

Node.JS: 15.1.0
EC2 Instance: c5.large
NestJS: 7.3.1

Answer 1

Different tops

What you're seeing is (likely) due to a difference in flavors of top.

I'm going to take a wild guess and say that your Docker image is perhaps based on Alpine? The top command in Alpine is busybox. It reports the per-process CPU usage as a percentage of the TOTAL number of CPUs available (nCPUs * 100%).

This differs from most other flavors of top, which report the per-process CPU usage as a percentage of a SINGLE CPU.

Both tops show same thing: ~50% usage on each CPU

The two top screenshots are actually showing the same thing: node process is using about 50% of each of the 2 CPUs.

Testing theory

We can test this with the following:

# This will max out 1 cpu of the system
docker run --name stress --rm -d alpine sh -c 'apk add stress-ng && stress-ng --cpu 1'

# This shows the busybox top with usage as ratio of total CPUs
# press 'c' in top to see the per-CPU info at the top
docker exec -it stress top

# This will install and run procps top, with usage as a ratio of single CPU
docker exec -it stress sh -c 'apk add procps && /usr/bin/top'

In the screenshot above, we can see two different flavors of top. They are reporting the same CPU usage, but the upper one reports this as "100% CPU" (as a percentage of a single core), while lower one reports this as 6% (1/16 cores = 6.25%).

What does this tell us about node's CPU usage?

Node is single-threaded, and cannot use more than 100% of a CPU. ...sort of. Under the hood, Node uses libuv, which does run threads in silos. This is how Node receives asynchronous events for IO operations, for example. These threads do use CPU and can push your CPU usage over 100%. Some packages are also written as add-ons to Node, and these also use threads.

The environment variable UV_THREADPOOL_SIZE limits the maximum number of libuv-controlled threads which may run simultaneously. Setting this to a larger number (default is 4) before running node may remove a bottleneck.

If you are doing some CPU-intensive operations, consider using cluster, Worker Threads, writing your own add-on or spawning separate processes to do the computation.