writing framed data without extra write() cost

Question

So I'm sending data on a TCP socket, prefixed with the size of data, as so:

write(socket, &length, sizeof(length));
write(socket, data, length);

(Note: I have wrapper writen functions as described in the Unix Network Programming book, and am checking for errors, etc. The above is just for the simplicity of this question).

Now, my experience is that breaking up data into multiple writes can cause significant slowdown. I have had success speeding things up by creating my own buffer, then sending out one big chunk.

However, in the above case data may be incredibly large (lets say 1 Gig). I don't want to create a buffer 1 Gig large + 4 bytes, just to be able to have one write() call. Is there any way of doing something akin to:

 write(socket, &length, data, sizeof(length) + length)

without paying the price of a large memory allocation ahead of time? I suppose I could just pre-allocate a chunk the size of write's buffer, and continuously send that (the below code has errors, namely, should be sending &chunk + 4 in some instances, but this is just the idea):

length += 4;

char chunk[buffer_size];
var total = 0;

while (total < length)
{
     if (total < 4)
     {
         memcpy(&chunk, &length, 4);
         total += 4;
     }

     memcpy(&chunk, data + total, min(buffer_size, length - total));
     write(sock, &chunk, min(buffer_size, length - total));

     total += min(buffer_size, length - total);
}

But in that case I don't know what write's buffer size actually is (is there an API to get it?) I also don't know if this is an appropriate solution.

Answer 1

There is an option to do this already. It will inform your network layer that you are going to send more data and you want to buffer rather than send it as soon as possible.

setsockopt(sock_descriptor, IPPROTO_TCP, TCP_CORK, (char *)&val, sizeof(val));

val is an int, and should be 0 or 1, with the "cork" on, your network layer will buffer things as much as possible, to only send full packets, you might want to "pop the cork" and "cork" again to handle the next batch of transmissions that you need to make on the socket.

Your idea is correct, this just saves you the trouble of implementing it, since it's already done in the network stack.