Partial read/write issue in sendmsg/recvmsg

Question

I am writing a program that passes file descriptors between two processes using sendmsg and recvmsg through domain socket. For sending a file descriptor, additional data is included in msghdr.msg_iov and msghdr.msg_iolen. However, I was informed that similar to normal read and write system call, sendmsg and recvmsg also have partial read/write problem. In this case, will the data in ancillary field be duplicated automatically for each partial data? I am asking this because my implementation requires non-blocking mechanism. Let me use the following example to elaborate it a bit more

Sender: send msghdr data which contains a fd in ancillary field and K bytes in msg_iov

Receiver: (1) partial read, K1 bytes (2) partial read, K-K1 bytes

Now as the above example, I should actually process the data after the step (2) when all data arrive. In this case, can I still correctly extract the fd from the ancillary field? Or does it only appear in the first partial read?

Answer 1

From a quick gander at the kernel source (linux, but see below), I believe it's up to you to make sure that the ancillary data only gets sent once. That is, in non-blocking mode, if there is no room in the receiving socket, you will get back EAGAIN/EWOULDBLOCK and neither data nor ancillary data will be sent. But if there is some space on the receiving side, then the initial portion of the data will be sent and the ancillary data will also be sent. You would then receive a return byte count indicating a partial send but the ancillary data will have been sent.

You need to be aware of that when you attempt the send the rest of your message, because the kernel maintains no memory that you've previously sent a partial buffer with which the subsequent buffer is logically contiguous (really no way it could -- you could be sending entirely different data for all it knows). So if you simply provide the same ancillary data for subsequent buffer parts, I believe the kernel will happily deliver the ancillary data again with your subsequent buffer part(s). This might result in duplicate file descriptors on the receiver side (which you would probably neglect to close since you wouldn't be expecting them) -- if you don't avoid it.

Now if you're in blocking mode on the sending side, and the transfer gets broken into multiple parts, the ancillary data will only be sent once -- with the first buffer part, because the sending of the entire buffer remains within kernel control.

On the receiving side, you would therefore need to be aware that the ancillary data accompanies the first chunk of received data, if you haven't received the entire logical message.

I believe this behavior is consistent with that reported in the stackexchange reference given by @Klas-Lindbäck (https://unix.stackexchange.com/questions/185011/what-happens-with-unix-stream-ancillary-data-on-partial-reads). (That question didn't deal with non-blocking-mode though.)

This answer is specific to linux. So it's certainly possible that results would differ slightly on other OSes, though it's difficult for me to see how they could be significantly different and still maintain sane semantics. The kernel can't reasonably maintain a memory of what has been sent previously and the sendmsg prototype doesn't allow it to overwrite the user's msghdr to reflect that the msg_control part has already been sent.