When should I use TCP_NODELAY and when TCP_CORK?

Question

First of all not both of them disables Nagle's algorithm.

Nagle's algorithm is for reducing more number of small network packets in wire. The algorithm is: if data is smaller than a limit (usually MSS), wait until receiving ACK for previously sent packets and in the mean time accumulate data from user. Then send the accumulated data.

if [ data > MSS ]
    send(data)
else
    wait until ACK for previously sent data and accumulate data in send buffer (data)
    And after receiving the ACK send(data)

This will help in applications like telnet. However, waiting for the ACK may increase latency when sending streaming data. Additionally, if the receiver implements the 'delayed ACK policy', it will cause a temporary deadlock situation. In such cases, disabling Nagle's algorithm is a better option.

So TCP_NODELAY is used for disabling Nagle's algorithm.

TCP_CORK aggressively accumulates data. If TCP_CORK is enabled in a socket, it will not send data until the buffer fills to a fixed limit. Similar to Nagle's algorithm, it also accumulates data from user but until the buffer fills to a fixed limit not until receiving ACK. This will be useful while sending multiple blocks of data. But you have to be more careful while using TCP_CORK.

Until 2.6 kernel, both of these options are mutually exclusive. But in later kernel, both of them can exist together. In such case, TCP_CORK will be given more preference.

Ref:

• http://baus.net/on-tcp_cork/
• http://ccr.sigcomm.org/archive/2001/jan01/ccr-200101-mogul.pdf

TCP_NODELAY

Used to disable Nagle's algorithm to improve TCP/IP networks and decrease the number of packets by waiting until an acknowledgment of previously sent data is received to send the accumulated packets.

//From the tcp(7) manual:

TCP_CORK (or TCP_NOPUSH in FreeBSD)

If set, don't send out partial frames. All queued partial frames are sent when the option is cleared again. This is useful for prepending headers before calling sendfile(2), or for throughput optimization. As currently implemented, there is a ** 200-millisecond ceiling** on the time for which output is corked by TCP_CORK. If this ceiling is reached, then queued data is automatically transmitted. This option can be combined with TCP_NODELAY only since Linux 2.5.71. This option should not be used in code intended to be portable.

It's an optimisation, so like any optimisation:

1. Do not use it
2. Wait until performance becomes a problem, then having determined that socket latency is definitely the cause of it, and testing proves that this will definitely fix it, AND this is the easiest way of fixing it, do it.

Basically the aim is to avoid having to send out several frames where a single frame can be used, with sendfile() and its friends.

So for example, in a web server, you send the headers followed by the file contents, the headers will be assembled in-memory, the file will then be sent directly by the kernel. TCP_CORK allows you to have the headers and the beginning of the file sent in a single frame, even with TCP_NODELAY, which would otherwise cause the first chunk to be sent out immediately.