TCP packets merged at network level

Question

Does anybody knows how and why a counter party would receive TCP packets merged instead of individually packages? I already set TCP Nodelay to true at socket level, but tcpdump still sees some packets as merged. After 4 successful packets sent with size of 310 bytes, I got 3 x 1400 bytes instead of 15 x 310 bytes. This is causing some important latency. Thanks.

http://www.2shared.com/photo/_bN9UEqR/tcpdump2.html

s = new Socket(host, port);
s.setTcpNoDelay(true);
s.getOutputStream().write(byteMsg); 
s.getOutputStream().flush()

Answer 1

TCP is a stream-based protocol. It doesn't preserve boundaries with respect to send/recv calls. The only thing guaranteed is that the concatenation of send's will be the same as the concatenation of recv's (under normal circumstances).

If you're implementing a custom protocol and need some way to split the data into multiple logical messages, you need an encoding for that.

A simple encoding is to encode each message as a 32-bit unsigned integer denoting the length of the message payload, followed by the actual message payload. Then, on the receiving side, properly decode the input according to this encoding. To do that, you will need a buffer that will store a partially received message. If manipulating raw integers is a problem, you can encode the length some other way, e.g. as a decimal number followed by a newline.

Answer 2

Coalescing can occur in many places

• sender application buffering
• sender OS buffering
• sender network adapter buffers
• router buffers
• receiver network adapter buffers
• receiver OS buffering
• receiver application buffers/queues

It appears from what you have said there is coalescing between the senders network adapter and the receiver's OS. (As tcp-no-delay instructs the OS not to buffer and tcpdump reads before the application)