Can someone give me a good explanation of 'send' behavior for non-blocking sockets?

Question

I have read the documentation at least 10 times now and have also read some 10 or so code snippets and full programs where non-blocking sockets are used for sending data. The problem is that some of the tutorials are either for beginners (Beejs f.i.) or are pretty sloppy in their assumptions; and those that are not are complicated are specialized code examples that don't explain why they do what they do. Even SO knowledge base doesn't exhaustively cover the entire gamut of send behavior, in my opinion. What I am after are details on f.e:

• What does return code of 0 indicate exactly, and is it worth checking errno then or should one just discard the connection without further investigation?
• Does getting a negative return value warrant closing a connection gone bad, or is it only so unless errno is EWOULDBLOCK, EAGAIN or EINTR (...others) ?
• Is it worth checking errno when return value is > 0? Apparently, the value indicates amount of data "sent" (in quotes because it's a long process really, right), but since the socket is non-blocking, does it mean one can issue another call right away, or, depending on errno again, one should wait for the next sending occasion (using select/poll/epoll) ?
• Basically, does one check the return value first and only then the errno value? Or maybe send sets errno on each call, return value regardless? That would make error checking somewhat easier...
• If one gets EINTR, what would be a good, robust behavior for a program to take? Simply record the state and retry on next send occasion, like with EWOULDBLOCK and EAGAIN?
• Does one check for both EWOULDBLOCK and EAGAIN? Can we trust both having the same value, or does it depend on the implementation?
• Does send return EMSGSIZE for stream sockets? If it doesn't, then no buffer size is too big, right?
• Can return value itself be equal to either of the known error codes?

If you could provide an example of robust non-blocking send code, it would be absolutely appreciated.

Answer 1

A lot of questions here:

• What does return code of 0 indicate exactly, and is it worth checking errno then or should one just discard the connection without further investigation?

On a POSIX system, send(2) can never return 0 unless you call it with a length arg of 0. Check the docs for your specific system to make sure it follows the POSIX spec

• Does getting a negative return value warrant closing a connection gone bad, or is it only so unless errno is EWOULDBLOCK, EAGAIN or EINTR (...others) ?

No, a -1 return value (the only possible negative return value) just means that no data was sent. You need to check errno to see WHY -- see the send(2) man page for a full listing of all the possible errno value and what they mean

• Is it worth checking errno when return value is > 0 ? Apparently, the value indicates amount of data "sent" (in quotes because it's a long process really, right), but since the socket is non-blocking, does it mean one can issue another call right away, or, depending on errno again, one should wait for the next sending occasion (using select/poll/epoll) ?

If send returns success (> 0), then errno will be unchanged and will contain whatever it had before (which is probably an error from some earlier system call).

• Basically, does one check the return value first and only then the errno value? Or maybe send sets errno on each call, return value regardless? That would make error checking somewhat easier...

Check the return value first and then errno if the return value is -1. If you really want to, you can set errno to 0 before the call and then check it afterwards

• If one gets EINTR, what would be a good, robust behavior for a program to take? Simply record the state and retry on next send occasion, like with EWOULDBLOCK and EAGAIN?

Well, the simplest is to disable interruption of system calls, in which case you'll never get an EINTR. Treating it the same as EWOULDBLOCK/EAGAIN is good too.

• Does one check for both EWOULDBLOCK and EAGAIN? Can we trust both having the same value, or does it depend on the implementation?

Depends on the implementation, though generally they're the same. Sometimes there's weirdness with SysV vs BSD emulation modes that might make them different and either might occur

• Does send return EMSGSIZE for stream sockets? If it doesn't, then no buffer size is too big, right?

Stream sockets don't have atomic messages and EMSGSIZE is only for atomic messages, so no, stream sockets can't return EMSGSIZE

• Can return value itself be equal to either of the known error codes?

The only error code is -1. Success is the number of bytes written, so if you could write 2^32-1 bytes on a 32-bit machine (or 2^64-1 on a 64 bit machine), it would be a problem, but you can't write that many bytes (and you'll generally get a EINVAL or EFAULT if you try).

Answer 2

On the EINTR and system calls:

• if you are using GLIBC, you do not need to worry about that, at least in context of the system calls. I got that from Glibc FAQ, grep for "Why don't signals interrupt system calls anymore?"

• if you are using LINUX, then you probably don't have to worry about the weird semantics of the connect() system call, that David Madore rants about here. Otherwise, be prepared for other than usuall behavior for asynchronous connect() call.

Answer 3

I'll try to answer your questions.

• A return value of 0 from send indicates that 0 bytes were sent. An error is indicated by a return value of -1. If you called send with a length of 0, a return of 0 should be expected. While a non-blocking socket should return -1 with an errno of EAGAIN or EWOULDBLOCK if it would block, I wouldn't be excessively surprised if some implementation returned 0 bytes written instead.
• EWOULDBLOCK, EAGAIN, and EINTR are errors on which you should retry, do not close the connection when receiving one of those. Other errors do indicate a problem that should probably result in a close.
• No, do not check errno after a successful library call (unless the documentation specifically states that you can do this for some reason; I'm not aware of any offhand that do this). Note that errno may not remain unchanged across a successful library call, as the call may have made other calls that returned errors that were expected and properly handled (e.g. a call could try to stat a file, fully expecting that it may not exist; errno would then be ENOENT even though there was no real error). If send returns a short write, you could try again (and possibly get EWOULDBLOCK/EAGAIN) or you could wait for the next select.
• Yes, check the return value first. errno tells you nothing useful if the call succeeded.
• On EINTR, you could try again immediately or you could wait for the next time through the select loop.
• You must check for both EAGAIN and EWOULDBLOCK; I suppose you could do #if EAGAIN == EWOULDBLOCK if performance is particularly critical (but remember, profile then optimize).
• It would all depend on the underlying protocol, but generally I would expect a streaming protocol to not have atomic messages (unless perhaps when using MSG_OOB). For TCP, any buffer size should be fine.
• It certainly is possible for the return value to equal any of the errno constants, but it means nothing. For example, on my system if 11 bytes were written the return value would be equal to EAGAIN.

HTH.