recv with MSG_NONBLOCK and MSG_WAITALL

Question

I want to use recv syscall with nonblocking flags MSG_NONBLOCK. But with this flag syscall can return before full request is satisfied. So,

• can I add MSG_WAITALL flag? Will it be nonblocking?
• or how should I rewrite blocking recv into the loop with nonblocking recv

Answer 1

For IPv4 TCP receives on Linux at least, MSG_WAITALL is ignored if MSG_NONBLOCK is specified (or the file descriptor is set to non-blocking).

From tcp_recvmsg() in net/ipv4/tcp.c in the Linux kernel:

if (copied >= target && !sk->sk_backlog.tail)
        break;

if (copied) {
        if (sk->sk_err ||
            sk->sk_state == TCP_CLOSE ||
            (sk->sk_shutdown & RCV_SHUTDOWN) ||
            !timeo ||
            signal_pending(current))
                break;

target in this cast is set to to the requested size if MSG_DONTWAIT is specified or some smaller value (at least 1) if not. The function will complete if:

1. Enough bytes have been copied
2. There's a socket error
3. The socket has been closed or shutdown
4. timeo is 0 (socket is set to non-blocking)
5. There's a signal pending for the process

To me this seems like it may be a bug in Linux, but either way it won't work the way you want. It looks like dec-vt100's solution will, but there is a race condition if you try to receive from the same socket in more than one process or thread.
That is, another recv() call by another thread/process could occur after your thread has performed a peek, causing your thread to block on the second recv().