Ubuntu 14.04 nc 100% CPU usage

Question

I am using Ubuntu 14.04 server (8 cores, 16 GB RAM) for hosting a PHP website, MySQL and Redis. PHP web and MySQL has very low traffic (MySQL: Queries per second avg: 0.825). Redis processes 8011 commands per second.

Today I have noticed that nc stays in the top of top:

8348 root      20   0   11224    764    624 R 100.0  0.0   2277:01 nc                                                                                                                            
8319 root      20   0   11224    760    624 R 100.0  0.0   2277:59 nc                                                                                                                        
8324 root      20   0   11224    764    624 R 100.0  0.0   2278:09 nc                                                                                                                        
8344 root      20   0   11224    760    624 R 100.0  0.0   2277:07 nc

Stracing nc gives:

root@host:/home/user# strace -p 8348
Process 8348 attached
poll([{fd=3, events=POLLIN}, {fd=-1}], 2, 1000) = 1 ([{fd=3, revents=POLLERR}])
poll([{fd=3, events=POLLIN}, {fd=-1}], 2, 1000) = 1 ([{fd=3, revents=POLLERR}])
poll([{fd=3, events=POLLIN}, {fd=-1}], 2, 1000) = 1 ([{fd=3, revents=POLLERR}])
intentionally cutted N lines from output

A quick lookup on man poll gives me info that poll waits for one of a set of file descriptors to become ready to perform I/O.

How do I find out what is happening to file descriptors (is it file descriptors issue?) and fix nc eating up 100% CPU?

Answer 1

We had a similar issue recently. We have a cron job that sends some redis stats to graphite over udp via netcat, and after bringing one of our graphite hosts down for a while last week, we noticed that CPU usage on our redis boxes skyrocketed. It appears to be a bug in netcat: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=752931

The command we were running was something like this:

echo "{redis_metric}" | nc -w 1 -u ${graphite_host} 8125

Using the 'quit' option (-q), as opposed to the 'timeout' option (-w), seems to fix the issue for us:

echo "{redis_metric}" | nc -q 1 -u ${graphite_host} 8125

Hope that helps!