Handling Incomplete write() Calls

Question

In Linux/Unix, the write() call may end up writing fewer bytes than requested:

The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setrlimit(2)), or the call was interrupted by a signal handler after having written less than count bytes. (See also pipe(7).)

The C standard library's fwrite() has the same behaviour. Most code I've seen ignores this possibility, choosing to handle errors in the following manner:

int ret = write(fd, buf, size);
if (ret < 0) {
    printf("Couldn't write %s: %s\n", path, strerror(errno));
    exit(1);
}

I've personally gotten in the habit of modifying the condition so that we are checking

if (ret != size) {
    printf("Couldn't write %s: %s\n", path, strerror(errno));
    exit(1);
}

Which notices this condition. However, I've also noticed that my program occasionally exits with:

Couldn't write /some/file: Success

I suppose this isn't too surprising. But then what is the standard, robust, clean way to handle this case? Clearly "silent data corruption"---which seems to be the behaviour of every tutorial ever---isn't OK. I could modify my code so that it specially detects this case and exits.

But the example provided in man 2 write is just an example. Are there other examples where retrying would be the way to go (EINTR is an example...)? How do I detect these, and more importantly, make sure I've handled every case? Isn't there a standard clean way to make these error handlers?

Answer 1

Write will return a negative number if nothing is written under two circumstances:

• A temporary error (e.g. EINTR, EAGAIN, and EWOULDBLOCK); the first of these can happen with any write, the second two (broadly) only on non-blocking I/O.

• A permanent error.

Normally you would want to retry the first, so the routine is to repeat the write if EINTR, EAGAIN or EWOULDBLOCK is returned (though I've seen argument against the latter).

For example:

ssize_t
write_with_retry (int fd, const void* buf, size_t size)
{
    ssize_t ret;
    do
    {
         ret = write(fd, buf, size);
    } while ((ret<0) && (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK));
    return ret;
}

Also note (from the man page) that write can return a number of bytes written less than you requested in the case of non-blocking I/O, or blocking I/O (as the linux man-page makes clear).

OS-X man-page extract:

When using non-blocking I/O on objects, such as sockets, that are subject to flow control, write() and writev() may write fewer bytes than requested; the return value must be noted, and the remainder of the operation should be retried when possible.

Linux man-page extract (my emphasis):

The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setrlimit(2)), or the call was interrupted by a signal handler after having written less than count bytes.

You would normally be handling those with select(), but to handle that case manually:

ssize_t
write_with_retry (int fd, const void* buf, size_t size)
{
    ssize_t ret;
    while (size > 0) {
        do
        {
             ret = write(fd, buf, size);
        } while ((ret < 0) && (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK));
        if (ret < 0)
            return ret;
        size -= ret;
        buf += ret;
    }
    return 0;
}

Answer 2

The (ret < 0) condition ignores short writes and only reports formal errors. This is arguably sloppy, but fixing it requires extra code — it is plausible to think that it would mean you should use a function wrapped around the write system call.

The (ret != size) condition is much more sensitive as a test, but the follow-up actions are complex enough that again, it needs to be a function wrapped around write().

In a generic wrapper function, you need to distinguish:

• ret > 0 && ret < size: in this case, you should loop and try to write the residue of the buffer. You could simply return the number of bytes actually written (cumulatively).

• ret == 0: you wrote zero bytes, but there wasn't an error. This could happen if you have the file descriptor open with O_NONBLOCK. You should probably have a strategy for dealing with this — but it is going to be context dependent. Is a sleep appropriate? Or do you just retry — and risk hammering the system? Do you just try a few times? You get 0 bytes written if you request 0 bytes to be written, but that would be covered under ret == size. Otherwise, it is not clear that ret == 0 is possible. For a generic wrapper, if this occurs, maybe the best approach is to return the 0.

• ret < 0: the chances are high that any retry will fail. Return the error condition. However, it is worth retrying if you get one of a selected few errors: EINTR if the write() is interrupted by a signal; EAGAIN for a write on a non-blocking file descriptor that would block — sometimes known as EWOULDBLOCK; maybe a few others but none of the normal errors for write() are plausible candidates.

Most serious Unix (or Linux) system programming books discuss this. For example:

• W Richard Stevens, Stephen A Rago Advanced Programming in the Unix Environment, 3rd Edn

• Marc J Rochkind Advanced Unix Programming, 2nd Edn