HTTP in C: Why do I have to call recv() multiple times?

Question

I've started to learn about Network programming in C (sockets), and Internet protocols.

Currently I'm focusing on HTTP.

During my (very) short experience with implementing HTTP client using C, I discovered that sometimes you have to call recv() multiple times in order to get the entire message from the server - even though the client didn't send a message to the server between those seperate calls to recv().

For example:

When I was trying to implement a HTTP client with one call of recv(), I got just the headers (I think, I'm still a newbie with HTTP). But when I called recv() twice, I got the headers in the first call, and the body (the html code) in the second call.

It is not a problem of a short buffer, because the buffer I was using was long enough to hold the entire message.

Why does it happen? What is the reason that the client has to call recv() multiple times, although there is no new data that was sent by the client? I thought that if the client doesn't send new data to the server, the call to recv() will bring the entire response of the server.

I don't think this is a problem with my code, but if you are going to ask for the code, I have no problem to post it here. I just think this is unnecessary, correct me if I was wrong.

I don't think this is relevant, but I'm using Winsock2.

Answer 1

TCP was designed to be stream oriented. The receiver doesn't know anything about sent messages/packets. All it sees is a simple stream of bytes.

So you think like "I sent a message", but recv() only received half of it. The fact is, you never sent a message. You sent a bunch of bytes, and you cannot expect all of them to be received by a single call. You could receive them in 1, 2 or many calls; you could receive the end of a previous "message" combined with the beginning of the next one and so on.

The only guarantees are that you will receive the bytes in the same order they were sent and you will never receive 0 bytes until the stream is closed. That's how the network API works and you have to get used to it.