Huge difference in netcat and iperf results for a 10G link

Question

I am confused to see huge difference between netcat and iperf results. I am having 10 G link connecting my server and client. I am getting around 10Gb/s for iperf but only ~280 MB/s for netcat. What can be the error ?

For Iperf

Server

iperf -s

Client

iperf -c 172.79.56.27 -i1 -t 10

Result:

Client connecting to 172.79.56.27, TCP port 5001

TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 172.79.56.28 port 46058 connected with 172.79.56.27 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  1.07 GBytes  9.23 Gbits/sec
[  3]  1.0- 2.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  2.0- 3.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  3.0- 4.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  4.0- 5.0 sec  1.09 GBytes  9.36 Gbits/sec
[  3]  5.0- 6.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  6.0- 7.0 sec  1.09 GBytes  9.36 Gbits/sec
[  3]  7.0- 8.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  8.0- 9.0 sec  1.09 GBytes  9.36 Gbits/sec
[  3]  9.0-10.0 sec  1.09 GBytes  9.35 Gbits/sec
[  3]  0.0-10.0 sec  10.9 GBytes  9.34 Gbits/sec

For netcat,

Server

nc -v -v -l -n 2222 >/dev/null

Client

time dd if=/dev/zero | nc -v -v -n 172.79.56.27 2222


Connection to 172.79.56.27 2222 port [tcp/*] succeeded!
^C6454690+0 records in
 6454690+0 records out
 3304801280 bytes (3.3 GB) copied, 11.4463 s, 289 MB/s


real    0m11.449s
user    0m6.868s
sys 0m15.372s

Answer 1

Mr user1352179,

Run your netcat test again and watch htop in another window. I bet you will see that the bottleneck here is the single thread of dd reading /dev/zero. Try running the test again in parallel with 'n' instances of dd|netcat where 'n' = the # of cores in your system. Then add the total bandwidth together from all parallel runs to see the real result. (Make sure you transmit on different ports & have your receiving end running multiple threads as well & listening on multiple ports).